Protocol Buffers over TCP/IP¶

This communication protocol allows languages that can communication over a TCP/IP connection to interact with a kRPC server.

Note

If a client library is available for your language, you do not need to implement this protocol.

Sending and Receiving Messages¶

Communication with the server is performed via Protocol Buffer messages, encoded according to the protobuf binary format. When sending messages to and from the server, they are prefixed with their size, in bytes, encoded as a Protocol Buffers varint.

To send a message to the server:

Encode the message using the Protocol Buffers format.

Get the length of the encoded message data (in bytes) and encode that as a Protocol Buffers varint.

Send the encoded length followed by the encoded message data.

To receive a message from the server, do the reverse:

Receive the size of the message as a Protocol Buffers encoded varint.

Decode the message size.

Receive message size bytes of message data.

Decode the message data.

Connecting to the RPC Server¶

A client invokes remote procedures by communicating with the RPC server. To establish a connection, a client must do the following:

Open a TCP socket to the server on its RPC port (which defaults to 50000).
Send a ConnectionRequest message to the server. This message is defined as:
message ConnectionRequest {
  Type type = 1;
  string client_name = 2;
  bytes client_identifier = 3;
  enum Type {
    RPC = 0;
    STREAM = 1;
  };
}
The type field should be set to ConnectionRequest.RPC and the client_name field can be set to the name of the client to display on the in-game UI. The client_identifier should be left blank.
Receive a ConnectionResponse message from the server. This message is defined as:
message ConnectionResponse {
  Status status = 1;
  enum Status {
    OK = 0;
    MALFORMED_MESSAGE = 1;
    TIMEOUT = 2;
    WRONG_TYPE = 3;
  }
  string message = 2;
  bytes client_identifier = 3;
}
If the status field is set to ConnectionResponse.OK then the connection was successful. If not, the message field contains a description of what went wrong.

When the connection is successful, the client_identifier contains a unique 16-byte identifier for the client. This is required when connecting to the stream server (described below).

Connecting to the Stream Server¶

Clients can receive Streams from the stream server. To establish a connection, a client must first connect to the RPC server (as above) then do the following:

Open a TCP socket to the server on its stream port (which defaults to 50001).
Send a ConnectionRequest message, with its type field set to ConnectionRequest.STREAM and its client_identifier field set to the value received in the client_identifier field of the ConnectionResponse message received when connecting to the RPC server earlier.
Receive a ConnectionResponse message, similarly to the RPC server, and check that the value of the status field is ConnectionResponse.OK. If not, then the connection was not successful, and the message field contains a description of what went wrong.

Connecting to the stream server is optional. If the client doesn’t require stream functionality, there is no need to connect.

Invoking Remote Procedures¶

See Messaging Protocol.

Examples¶

The following Python code connects to the RPC server at address 127.0.0.1 and port 50000 using the name “Jeb”. Next, it connects to the stream server on port 50001. It then invokes the KRPC.GetStatus RPC, receives and decodes the result and prints out the server version number from the response.

The following python code connects to the RPC server at address 127.0.0.1 and port 50000, using the name “Jeb”. Next, it connects to the stream server on port 50001. Finally it invokes the KRPC.GetStatus procedure, and receives, decodes and prints the result.

To send and receive messages to the server, they need to be encoded and decoded from their binary format:

The encode_varint and decode_varint functions convert between Python integers and Protocol Buffer varint encoded integers.

send_message encodes a message, sends the length of the message to the server as a Protocol Buffer varint encoded integer, and then sends the message data.

recv_message receives the size of the message, decodes it, receives the message data, and decodes it.

import socket
from google.protobuf.internal.encoder import _VarintEncoder
from google.protobuf.internal.decoder import _DecodeVarint
from krpc.schema import KRPC_pb2 as KRPC


def encode_varint(value):
    """ Encode an int as a protobuf varint """
    data = []
    _VarintEncoder()(data.append, value, False)
    return b''.join(data)


def decode_varint(data):
    """ Decode a protobuf varint to an int """
    return _DecodeVarint(data, 0)[0]


def send_message(conn, msg):
    """ Send a message, prefixed with its size, to a TPC/IP socket """
    data = msg.SerializeToString()
    size = encode_varint(len(data))
    conn.sendall(size + data)


def recv_message(conn, msg_type):
    """ Receive a message, prefixed with its size, from a TCP/IP socket """
    # Receive the size of the message data
    data = b''
    while True:
        try:
            data += conn.recv(1)
            size = decode_varint(data)
            break
        except IndexError:
            pass
    # Receive the message data
    data = conn.recv(size)
    # Decode the message
    msg = msg_type()
    msg.ParseFromString(data)
    return msg


# Open a TCP/IP socket to the RPC server
rpc_conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
rpc_conn.connect(('127.0.0.1', 50000))

# Send an RPC connection request
request = KRPC.ConnectionRequest()
request.type = KRPC.ConnectionRequest.RPC
request.client_name = 'Jeb'
send_message(rpc_conn, request)

# Receive the connection response
response = recv_message(rpc_conn, KRPC.ConnectionResponse)

# Check the connection was successful
if response.status != KRPC.ConnectionResponse.OK:
    raise RuntimeError('Connection failed: ' + response.message)
print('Connected to RPC server')

# Invoke the KRPC.GetStatus RPC
call = KRPC.ProcedureCall()
call.service = 'KRPC'
call.procedure = 'GetStatus'
request = KRPC.Request()
request.calls.extend([call])
send_message(rpc_conn, request)

# Receive the response
response = recv_message(rpc_conn, KRPC.Response)

# Check for an error in the response
if response.HasField('error'):
    raise RuntimeError('ERROR: ' + str(response.error))

# Check for an error in the results
assert len(response.results) == 1
if response.results[0].HasField('error'):
    raise RuntimeError('ERROR: ' + str(response.results[0].error))

# Decode the return value as a Status message
status = KRPC.Status()
status.ParseFromString(response.results[0].value)

# Print out the Status message
print(status)

The following example demonstrates how to set up and receive data from the server over a stream:

import binascii
import socket
from google.protobuf.internal.encoder import _VarintEncoder
from google.protobuf.internal.decoder import _DecodeVarint
from krpc.schema import KRPC_pb2 as KRPC


def encode_varint(value):
    """ Encode an int as a protobuf varint """
    data = []
    _VarintEncoder()(data.append, value, False)
    return b''.join(data)


def decode_varint(data):
    """ Decode a protobuf varint to an int """
    return _DecodeVarint(data, 0)[0]


def send_message(conn, msg):
    """ Send a message, prefixed with its size, to a TPC/IP socket """
    data = msg.SerializeToString()
    size = encode_varint(len(data))
    conn.sendall(size + data)


def recv_message(conn, msg_type):
    """ Receive a message, prefixed with its size, from a TCP/IP socket """
    # Receive the size of the message data
    data = b''
    while True:
        try:
            data += conn.recv(1)
            size = decode_varint(data)
            break
        except IndexError:
            pass
    # Receive the message data
    data = conn.recv(size)
    # Decode the message
    msg = msg_type()
    msg.ParseFromString(data)
    return msg


# Open a TCP/IP socket to the RPC server
rpc_conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
rpc_conn.connect(('127.0.0.1', 50000))

# Send an RPC connection request
request = KRPC.ConnectionRequest()
request.type = KRPC.ConnectionRequest.RPC
request.client_name = 'Jeb'
send_message(rpc_conn, request)

# Receive the connection response
response = recv_message(rpc_conn, KRPC.ConnectionResponse)

# Check the connection was successful
if response.status != KRPC.ConnectionResponse.OK:
    raise RuntimeError('Connection failed: ' + response.message)
print('Connected to RPC server')

# Print out the clients identifier
print('RPC client idenfitier = %s' % binascii.hexlify(
    bytearray(response.client_identifier)).decode('utf8'))

# Open a TCP/IP socket to the Stream server
stream_conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
stream_conn.connect(('127.0.0.1', 50001))

# Send a stream connection request, containing the clients identifier
request = KRPC.ConnectionRequest()
request.type = KRPC.ConnectionRequest.STREAM
request.client_identifier = response.client_identifier
send_message(stream_conn, request)

# Receive the connection response
response = recv_message(stream_conn, KRPC.ConnectionResponse)

# Check the connection was successful
if response.status != KRPC.ConnectionResponse.OK:
    raise RuntimeError("Connection failed: " + response.message)
print('Connected to stream server')

# Build a KRPC.GetStatus call to be streamed
stream_call = KRPC.ProcedureCall()
stream_call.service = 'KRPC'
stream_call.procedure = 'GetStatus'

# Call KRPC.AddStream to add the stream
call = KRPC.ProcedureCall()
call.service = 'KRPC'
call.procedure = 'AddStream'
arg = KRPC.Argument()
arg.position = 0
arg.value = stream_call.SerializeToString()
call.arguments.extend([arg])
request = KRPC.Request()
request.calls.extend([call])
send_message(rpc_conn, request)

# Receive the response
response = recv_message(rpc_conn, KRPC.Response)

# Check for an error in the response
if response.HasField('error'):
    raise RuntimeError('ERROR: ' + str(response.error))

# Check for an error in the results
assert len(response.results) == 1
if response.results[0].HasField('error'):
    raise RuntimeError('ERROR: ' + str(response.results[0].error))

# Decode the return value as a Stream message
stream = KRPC.Stream()
stream.ParseFromString(response.results[0].value)

# Repeatedly receive stream updates from the stream server
while True:
    update = recv_message(stream_conn, KRPC.StreamUpdate)
    assert len(update.results) == 1
    assert stream.id == update.results[0].id
    # Decode and print the return value
    status = KRPC.Status()
    status.ParseFromString(update.results[0].result.value)
    print(status)