Skip to content

OSC

This guide delves into how to send tracking data to MANUS Core using the OSC (Open Sound Control) protocol. OSC is a lightweight messaging protocol designed for simple and fast real-time communication over networks.

OSC Format

An OSC message consists of three parts:

  • Address Pattern This is a string that identifies what the message represents. In our case this will generally be a tracker. For example: /my_tracking_system/joint/right
  • Type Tag This is a string which defines the data types the message contains. Common types include f for floats and s for strings. For example our tracker messages expect 7 floats so the type tag would be fffffff.
  • Payload This is the actual data being sent. In this example the total message could be: /my_tracking_system/joint/right fffffff 10.0 15.0 20.0 0.0 0.0 0.0 1.0.

How messages are sent

A sender transmits OSC packets to a specific IP address and port, and a receiver (like MANUS Core) listens on that port for incoming messages. By default, MANUS Core listens on port 8000.

For more information, see the spec https://opensoundcontrol.stanford.edu/index.html.

When to use OSC

OSC allows tracking data to be sent without requiring the SDK app to directly integrate the tracking system. This is especially useful when using SDK Integrated.

If you want to send the tracker data to MANUS Core using SDK calls instead, see the Custom trackers article.

SDK Integrated

The OSC tracking system should be enabled through use of the CoreSdk_SetTrackerSystemsSettings call. For more information on this, see the SDKClient. This call is also used to set the listening port.

  • Trackers can be assigned by use of the address. This is done by setting the id value to left or right. This will assign the glove to the left or right hand of the user at index 0.
  • Alternatively, trackers can be assigned through use of SDK calls. For more information on this, see the SDKClient.

SDK Remote

In SDK Remote, either the CoreSdk_SetTrackerSystemsSettings call or the Dashboard can be used to enable OSC and to set the listening port.

Please see the following pages on how to set OSC tracking up in the Dashboard:

  • For information on how to enable OSC and set the listening port, see the Dashboard Settings.
  • For information on how to assign the trackers, see the User Settings.
  • If necessary, an offset can also be set here. For information on this, see Tracker Offsets.

Message formats

The SDK accepts two kinds of OSC messages. Messages that contain tracker data and messages that contain information about the coordinate space of the tracking system.

Tracker messages

The address of the data is in the /source/type/(side|id) format.

Container Description
source Specifies which tracking system is being used.
type Specifies whether the tracking data tracks the wrist itself or a point located on the MANUS Universal Mounting System. This is done by setting either joint or tracker here respectively.
side Can be used to specify the side of the tracker for SDK Integrated. In this case the value should be "left" or "right". When not using this to set the side, this container is used as id instead.
id Unique identifier that can be any string of choice, as long as it's unique.

For example, When using SDK Integrated the address could be /my_tracking_system/joint/right. When using Core the address could be /my_other_tracking_system/tracker/123.

The packet should consist of 7 floats. These represent in order:

  • Position X
  • Position Y
  • Position Z
  • Rotation X
  • Rotation Y
  • Rotation Z
  • Rotation W

The example below builds a tracker message with the address /source/type/side and a payload of 7 floats (position xyz, rotation xyzw). Note that strings are null-terminated and padded to a 4-byte boundary, and floats are written in network (big-endian) byte order:

using Buffer = std::vector<char>;

// Append raw bytes to the buffer.
static void writeRaw(Buffer& buffer, const void* data, size_t size) {
    auto bytes = static_cast<const char*>(data);
    buffer.insert(buffer.end(), bytes, bytes + size);
}

// Pad the buffer with zero bytes until its size is a multiple of 4 (OSC alignment).
static void padTo4ByteBoundary(Buffer& buffer) {
    while (buffer.size() % 4 != 0) {
        buffer.push_back('\0');
    }
}

// Append a null-terminated string and pad to a 4-byte boundary.
static void writeString(Buffer& buffer, const std::string& value) {
    writeRaw(buffer, value.data(), value.size());
    buffer.push_back('\0');
    padTo4ByteBoundary(buffer);
}

// Append a 32-bit float in network (big-endian) byte order.
static void writeFloat32(Buffer& buffer, float value) {
    uint32_t raw;
    std::memcpy(&raw, &value, sizeof(raw));
    raw = htonl(raw);
    writeRaw(buffer, &raw, sizeof(raw));
}

// OSC message: "/source/type/side" + 7 floats (position xyz, rotation xyzw).
static Buffer buildTrackerMessage(const std::string& source,
                                  const std::string& type,
                                  const std::string& side,
                                  float px, float py, float pz,
                                  float rx, float ry, float rz, float rw) {
    Buffer buffer;

    // Address pattern, e.g. "/sample/joint/right".
    writeString(buffer, "/" + source + "/" + type + "/" + side);

    // Type tag string: a leading ',' followed by one character per argument.
    writeString(buffer, ",fffffff");

    // Payload: 7 floats describing position and rotation.
    writeFloat32(buffer, px);
    writeFloat32(buffer, py);
    writeFloat32(buffer, pz);
    writeFloat32(buffer, rx);
    writeFloat32(buffer, ry);
    writeFloat32(buffer, rz);
    writeFloat32(buffer, rw);

    return buffer;
}

import struct


# Append raw bytes to the buffer.
def write_raw(buffer: bytearray, data: bytes) -> None:
    buffer.extend(data)


# Pad the buffer with zero bytes until its size is a multiple of 4 (OSC alignment).
def pad_to_4_byte_boundary(buffer: bytearray) -> None:
    while len(buffer) % 4 != 0:
        buffer.append(0)


# Append a null-terminated string and pad to a 4-byte boundary.
def write_string(buffer: bytearray, value: str) -> None:
    write_raw(buffer, value.encode("utf-8"))
    buffer.append(0)
    pad_to_4_byte_boundary(buffer)


# Append a 32-bit float in network (big-endian) byte order.
def write_float32(buffer: bytearray, value: float) -> None:
    write_raw(buffer, struct.pack(">f", value))


# OSC message: "/source/type/side" + 7 floats (position xyz, rotation xyzw).
def build_tracker_message(source, type_, side, px, py, pz, rx, ry, rz, rw) -> bytearray:
    buffer = bytearray()

    # Address pattern, e.g. "/sample/joint/right".
    write_string(buffer, f"/{source}/{type_}/{side}")

    # Type tag string: a leading ',' followed by one character per argument.
    write_string(buffer, ",fffffff")

    # Payload: 7 floats describing position and rotation.
    write_float32(buffer, px)
    write_float32(buffer, py)
    write_float32(buffer, pz)
    write_float32(buffer, rx)
    write_float32(buffer, ry)
    write_float32(buffer, rz)
    write_float32(buffer, rw)

    return buffer

Coordinate system messages

The coordinate system message is used to set the coordinate space the tracking data is in. MANUS Core uses this to correctly interpret the tracking data.

  • The address of the data should be /coordsystem.
  • The packet itself consists of a handedness, up-axis, forward-axis and scale in that order.
Property Description
handedness The handedness of the coordinate system. Should be set to right-handed or left-handed.
up-axis The axis that is considered up. Should be set to x-up, "x-down, "y-up, y-down, z-up or z-down.
forward-axis The axis that is considered forward. Should be set to x-positive, x-negative, y-positive, y-negative, z-positive or z-negative.
scale A float value representing how many meters one unit corresponds to. For example, a value of 0.1 means that a value of 10 would be interpreted as 1 meter.

The example below builds a coordinate system message with the address /coordsystem and a payload describing the handedness, up axis, forward (view) axis, and scale. It reuses the writeString and writeFloat32 helpers shown above:

// OSC message: "/coordsystem" + handedness, up axis, view axis, scale.
static Buffer buildCoordinateSystemMessage(const std::string& handedness,
                                           const std::string& upAxis,
                                           const std::string& viewAxis,
                                           float scale) {
    Buffer buffer;

    // Address pattern.
    writeString(buffer, "/coordsystem");

    // Type tag string: three strings followed by one float.
    writeString(buffer, ",sssf");

    // Payload.
    writeString(buffer, handedness);
    writeString(buffer, upAxis);
    writeString(buffer, viewAxis);
    writeFloat32(buffer, scale);

    return buffer;
}

# OSC message: "/coordsystem" + handedness, up axis, view axis, scale.
def build_coordinate_system_message(handedness, up_axis, view_axis, scale) -> bytearray:
    buffer = bytearray()

    # Address pattern.
    write_string(buffer, "/coordsystem")

    # Type tag string: three strings followed by one float.
    write_string(buffer, ",sssf")

    # Payload.
    write_string(buffer, handedness)
    write_string(buffer, up_axis)
    write_string(buffer, view_axis)
    write_float32(buffer, scale)

    return buffer

Bundles

Multiple OSC messages can be grouped into a bundle. When tracker messages are grouped in a bundle, it ensures that the SDK receives and processes all the contained tracker data simultaneously.

A bundle starts with the string #bundle, followed by an 8-byte timetag (a value of 0 means "process immediately"), and then a sequence of [size, message] entries. The example below reuses the writeString and writeRaw helpers shown later:

// Append a 32-bit signed integer in network (big-endian) byte order.
static void writeInt32(Buffer& buffer, int32_t value) {
    value = htonl(value);
    writeRaw(buffer, &value, sizeof(value));
}

// OSC bundle: "#bundle" + 8-byte timetag (0 = immediate) + [size, message]...
static Buffer buildOscBundle(const std::vector<Buffer>& messages) {
    Buffer buffer;

    // Bundle identifier.
    writeString(buffer, "#bundle");

    // 8-byte timetag. A value of 0 tells the receiver to process immediately.
    for (int i = 0; i < 8; ++i) {
        buffer.push_back('\0');
    }

    // Each contained message is prefixed with its size as an int32.
    for (const Buffer& message : messages) {
        writeInt32(buffer, static_cast<int32_t>(message.size()));
        writeRaw(buffer, message.data(), message.size());
    }

    return buffer;
}

# Append a 32-bit signed integer in network (big-endian) byte order.
def write_int32(buffer: bytearray, value: int) -> None:
    write_raw(buffer, struct.pack(">i", value))


# OSC bundle: "#bundle" + 8-byte timetag (0 = immediate) + [size, message]...
def build_osc_bundle(messages) -> bytearray:
    buffer = bytearray()

    # Bundle identifier.
    write_string(buffer, "#bundle")

    # 8-byte timetag. A value of 0 tells the receiver to process immediately.
    for _ in range(8):
        buffer.append(0)

    # Each contained message is prefixed with its size as an int32.
    for message in messages:
        write_int32(buffer, len(message))
        write_raw(buffer, message)

    return buffer

Sample of flow

We have created a sample showing how to send OSC messages to MANUS Core. The snippet below shows the typical flow: open a UDP socket targeting the MANUS Core listening port (default 8000), send a single coordinate system message, then continuously send tracker messages each frame. Make sure MANUS Core is running and listening at the correct port at the time the coordinate system message is sent.

int main() {
    sockaddr_in dst{};
    dst.sin_family = AF_INET;
    dst.sin_port = htons(8000);
    dst.sin_addr.s_addr = inet_addr("127.0.0.1");

    int sock = createUdpSocket();

    sendOscMessage(sock, buildCoordinateSystemMessage("right-handed", "y-up", "z-positive", 1.0f), dst);
    std::cout << "Sent coordinate system message." << std::endl;

    float angle = 0.0f;

    while (true) {
        auto left    = buildTrackerMessage("sample", "joint", "left",   std::cos(angle),        2.0f, std::sin(angle),          0,     0,     0,     1    );
        auto right   = buildTrackerMessage("sample", "joint", "right",  std::cos(angle + 2.0f), 2.0f, std::sin(angle + 2.0f),   0,     0.707f, 0,    0.707f);
        auto another = buildTrackerMessage("sample", "joint", "another",std::cos(angle + 4.0f), 2.0f, std::sin(angle + 4.0f),   0,     1,     0,     0    );

        angle += 0.05f;

        sendOscMessage(sock, left,    dst);
        sendOscMessage(sock, right,   dst);
        sendOscMessage(sock, another, dst);

        // Alternatively, send all messages in a single bundle:
        // sendOscMessage(sock, buildOscBundle({ left, right, another }), dst);

    #if defined(_WIN32)
        Sleep(10);
    #else
        usleep(10000);
    #endif
    }
    return 0;
}

static void sendOscMessage(int sock, const Buffer& msg, const sockaddr_in& dst) {
    sendto(sock, msg.data(), (int)msg.size(), 0, reinterpret_cast<const sockaddr*>(&dst), sizeof(dst));
}

def create_udp_socket() -> socket.socket:
    return socket.socket(socket.AF_INET, socket.SOCK_DGRAM)


def send_osc_message(sock: socket.socket, msg: bytearray, dst) -> None:
    sock.sendto(bytes(msg), dst)


def main():
dst = ("127.0.0.1", 8000)
sock = create_udp_socket()

send_osc_message(sock, build_coordinate_system_message("right-handed", "y-up", "z-positive", 1.0), dst)
print("Sent coordinate system message.")

angle = 0.0

while True:
    left    = build_tracker_message("sample", "joint", "left",    math.cos(angle),        2.0, math.sin(angle),          0,      0,      0,      1     )
    right   = build_tracker_message("sample", "joint", "right",   math.cos(angle + 2.0),  2.0, math.sin(angle + 2.0),    0,      0.707,  0,      0.707 )
    another = build_tracker_message("sample", "joint", "another", math.cos(angle + 4.0),  2.0, math.sin(angle + 4.0),    0,      1,      0,      0     )

    angle += 0.05

    send_osc_message(sock, left,    dst)
    send_osc_message(sock, right,   dst)
    send_osc_message(sock, another, dst)

    # Alternatively, send all messages in a single bundle:
    # send_osc_message(sock, build_osc_bundle([left, right, another]), dst)

    time.sleep(0.01)


if __name__ == "__main__":
    main()

Troubleshooting

Not receiving data

  1. Check firewall: Ensure UDP port is not blocked
  2. Check port: Verify sender and MANUS Core are using the same port
  3. Check message format: Ensure messages match the expected format exactly - Tracker messages: /{SOURCE}/tracker/{SIDE} x y z qx qy qz qw (7 float arguments) - Joint messages: /{SOURCE}/joint/{SIDE} x y z qx qy qz qw (7 float arguments) - Coordinate system: /coordsystem handedness up-axis forward-axis scale (4 arguments)

Incorrect tracker positions/rotations

  1. Check coordinate system: Send a /coordsystem message to configure the source coordinate system
  2. Check quaternion order: Ensure quaternions are sent as (x, y, z, w)
  3. Check scale: Set the correct scale factor (1.0 for meters, 0.001 for millimeters, etc.)
  4. Check axis directions: Verify up-axis and forward-axis match your source system