License requirement

The functionality described requires a MANUS Bodypack or a MANUS license key with the SDK feature enabled.

SDK Minimal Client example

Introduction

The SDK Minimal Client is a Python example that demonstrates the basic functionality of the MANUS SDK. It will go over what is minimally required to get the SDK up and running and demonstrate how to receive raw skeleton data from all connected gloves. The SDK Minimal Client covers the following steps:

1. Initialization:

   • The SDK can be initialized for use with Core or Core Integrated using either the CoreSdk_InitializeCore() or CoreSdk_InitializeIntegrated() function respectively.
   • The coordinate system for the client is set using the CoreSdk_InitializeCoordinateSystemWithVUH() function.

2. Connection:

   • When not running in integrated mode, the client attempts to connect to a MANUS Core instance.
   • The connect() function demonstrates host discovery and connection.
   • Once connected, the client sets the RawSkeletonHandMotion to auto.
   • In the main loop, it checks for new raw skeleton data and prints node information.

3. Raw Skeleton Stream Callback:

   • The callback demonstrates how to interpret raw skeleton data from MANUS Core.
   • It prints the position and rotation of nodes and displays raw skeleton node information.

Overview

The SDK Minimal Client is built using direct CFFI calls to the MANUS SDK for maximum performance and minimal overhead. It demonstrates the most efficient way to use the Python SDK.

Location: examples/minimal_client.py

Features:

• Minimal dependencies and straightforward code
• Supports all three connection modes (Integrated, Local, Remote)
• Direct CFFI library calls (no wrapper overhead)
• Raw skeleton data streaming
• Node information display
• Thread-safe callback handling

Running the Example

python examples/minimal_client.py

Connection Mode Selection

On startup, you'll be prompted to select a connection mode:

Select what mode you would like to start in (and press enter to submit)
[1] Core Integrated - This will run standalone without the need for a MANUS Core connection
[2] Core Local - This will connect to a MANUS Core running locally on your machine
[3] Core Remote - This will search for a MANUS Core running locally on your network
Enter choice: 2

• [1] Integrated: No MANUS Core needed, runs standalone with integrated SDK
• [2] Local: Connects to localhost:7567 (requires local MANUS Core running)
• [3] Remote: Searches network for MANUS Core instances (takes ~3 seconds)

Remote Mode Host Selection

If choosing Remote mode and multiple MANUS Core instances are found:

Select which host you want to connect to (and press enter to submit)
[1] Office-Desktop (192.168.1.100)
[2] Lab-Workstation (192.168.1.105)
Enter choice: 1
Minimal client is connected, setting up skeletons.
Press SPACE to exit...

Operation

Once connected, the client:

1. Sets up the coordinate system (Z-up, X-right, right-handed)
2. Registers for raw skeleton data callbacks
3. Sets hand motion mode to auto
4. Enters the main loop receiving skeleton data
5. Displays skeleton frames and node information

Press SPACE to exit gracefully.

Initialization

Before using any functionality of the SDK, it must be initialized. This ensures the system is set up correctly and ready for use.

The app first prompts the user for the desired connection mode, then initializes the SDK accordingly.

Initialize SDK

def initialize_sdk(self) -> bool:
    """Initialize the SDK and set up connection"""
    # Prompt user for connection mode
    print("Select what mode you would like to start in (and press enter to submit)")
    print("[1] Core Integrated - This will run standalone without the need for a MANUS Core connection")
    print("[2] Core Local - This will connect to a MANUS Core running locally on your machine")
    print("[3] Core Remote - This will search for a MANUS Core running locally on your network")

    choice = input("Enter choice: ").strip()

    if choice == '1':
        self.connection_type = ConnectionType.Integrated
    elif choice == '2':
        self.connection_type = ConnectionType.Local
    elif choice == '3':
        self.connection_type = ConnectionType.Remote
    else:
        print("Invalid input, try again")
        return self.initialize_sdk()

    # Initialize SDK
    if self.connection_type == ConnectionType.Integrated:
        result = lib.CoreSdk_InitializeIntegrated()
    else:
        result = lib.CoreSdk_InitializeCore()

    if result != SDKReturnCode.Success:
        print(f"Failed to initialize SDK: {SDKReturnCode(result).name}")
        return False

    # Register callbacks
    return self.register_all_callbacks()

After initializing the SDK, callbacks are registered. For this minimal example, only the raw skeleton callback is registered:

Register Callbacks

def register_all_callbacks(self) -> bool:
    """Register SDK callbacks"""
    result = self.sdk.register_raw_skeleton_callback(
        self.on_raw_skeleton_stream_callback
    )

    if result != SDKReturnCode.Success:
        print(f"Failed to register callback: {SDKReturnCode(result).name}")
        return False

    return True

Next, the coordinate system is set up. This example uses a Z-up, X-positive, right-handed coordinate system:

Initialize Coordinate System

# Create coordinate system struct
coord_system = ffi.new("CoordinateSystemVUH*")
coord_system.view = AxisView.XFromViewer
coord_system.up = AxisPolarity.PositiveZ
coord_system.handedness = Side.Right
coord_system.unitScale = 1.0

# Initialize coordinate system
result = lib.CoreSdk_InitializeCoordinateSystemWithVUH(coord_system[0], True)

if result != SDKReturnCode.Success:
    print(f"Failed to initialize coordinate system: {SDKReturnCode(result).name}")
    return False

Connection

The connection process varies depending on the selected mode. For Integrated mode, minimal setup is needed. For Local or Remote modes, the SDK must discover and connect to a MANUS Core instance.

Host Discovery and Connection

Connect to Host

def connect(self) -> bool:
    """Connect to MANUS Core"""
    loopback_only = self.connection_type == ConnectionType.Local

    # Search for available hosts
    # - Local: 1ms timeout (only checks localhost)
    # - Remote: 1000ms timeout (network search)
    result = lib.CoreSdk_LookForHosts(1, loopback_only)
    if result != SDKReturnCode.Success:
        return False

    # Get number of hosts found
    count = ffi.new("uint32_t*")
    result = lib.CoreSdk_GetNumberOfAvailableHostsFound(count)
    num_hosts = count[0]
    if result != SDKReturnCode.Success or num_hosts == 0:
        return False

    # Fetch host list
    hosts_array = ffi.new("ManusHost[]", num_hosts)
    result = lib.CoreSdk_GetAvailableHostsFound(hosts_array, num_hosts)
    if result != SDKReturnCode.Success:
        return False

    # User selects host (if multiple and not local mode)
    host_selection = 0
    if not loopback_only and num_hosts > 1:
        print("Select which host you want to connect to (and press enter to submit)")
        for i in range(num_hosts):
            host = hosts_array[i]
            host_name = ffi.string(host.hostName).decode('utf-8')
            ip = ffi.string(host.ipAddress).decode('utf-8')
            print(f"[{i+1}] {host_name} ({ip})")

        try:
            choice = int(input("Enter choice: "))
            if choice < 1 or choice > num_hosts:
                return False
            host_selection = choice - 1
        except ValueError:
            return False

    # Connect to selected host
    result = lib.CoreSdk_ConnectToHost(hosts_array[host_selection])
    return result == SDKReturnCode.Success

Connection Retry Loop

The main run function uses a retry loop to handle connection failures:

Connection Retry Loop

def run(self):
    """Main loop"""
    # First connect to Core
    if self.connection_type == ConnectionType.Integrated:
        print("Minimal client is running in integrated mode.")
    else:
        print("Minimal client is connecting to MANUS Core. (make sure it is running)")

    # Try to connect
    while not self.connect():
        print("Minimal client could not connect. Trying again in a second.")
        time.sleep(1.0)

    if self.connection_type != ConnectionType.Integrated:
        print("Minimal client is connected, setting up skeletons.")

    # Set hand motion mode
    result = lib.CoreSdk_SetRawSkeletonHandMotion(HandMotion.Auto)
    if result != SDKReturnCode.Success:
        print(f"Failed to set hand motion mode: {SDKReturnCode(result).name}")

    print("Press SPACE to exit...")

    # Main loop
    while self.running:
        # Check for new skeleton data
        with self.skeleton_mutex:
            if self.next_raw_skeleton is not None:
                self.raw_skeleton = self.next_raw_skeleton
                self.next_raw_skeleton = None

        # Process skeleton data
        if self.raw_skeleton and len(self.raw_skeleton) > 0:
            print(f"Raw skeleton data obtained for frame: {self.frame_counter}")
            self.print_raw_skeleton_node_info()
            self.frame_counter += 1

        # Sleep ~30 FPS
        time.sleep(0.033)

        # Check for space key press (Windows)
        if msvcrt.kbhit():
            key = msvcrt.getch()
            if key == b' ':
                self.running = False

Raw Skeleton Stream Callback

Callbacks are the primary mechanism for receiving data from the SDK. The raw skeleton callback is called whenever new skeleton data becomes available.

Callback Registration

Register Raw Skeleton Callback

# In the ManusSDK wrapper class
result = sdk.register_raw_skeleton_callback(on_raw_skeleton_callback)

Callback Implementation

Raw Skeleton Callback Handler

def on_raw_skeleton_stream_callback(self, stream_info):
    """Callback for raw skeleton stream data"""
    try:
        skeletons_count = stream_info.skeletonsCount
        publish_time = stream_info.publishTime

        new_skeletons = []

        for i in range(skeletons_count):
            # Get skeleton info (metadata)
            info = ffi.new("RawSkeletonInfo*")
            result = lib.CoreSdk_GetRawSkeletonInfo(i, info)
            if result != SDKReturnCode.Success:
                continue

            # Get skeleton nodes (transforms)
            nodes_array = ffi.new("SkeletonNode[]", info[0].nodesCount)
            result = lib.CoreSdk_GetRawSkeletonData(i, nodes_array, info[0].nodesCount)
            if result != SDKReturnCode.Success:
                continue

            # Create skeleton object
            skeleton = ClientRawSkeleton()
            skeleton.info = info[0]
            skeleton.info.publishTime = publish_time
            skeleton.nodes = [nodes_array[j] for j in range(info[0].nodesCount)]

            new_skeletons.append(skeleton)

        # Store new skeleton data (thread-safe)
        with self.skeleton_mutex:
            self.next_raw_skeleton = new_skeletons

    except Exception as e:
        print(f"Error in skeleton callback: {e}")

Data Processing

After receiving skeleton data in the callback, process it in your main thread:

Process Raw Skeleton Data

def print_raw_skeleton_node_info(self):
    """Print skeleton node information"""
    if not self.raw_skeleton or len(self.raw_skeleton) == 0:
        return

    skeleton = self.raw_skeleton[0]

    # Print first node position and rotation
    if skeleton.nodes and len(skeleton.nodes) > 0:
        node = skeleton.nodes[0]
        pos = node.transform.position
        rot = node.transform.rotation
        print(f"  Node 0 Position: x={pos.x:.3f} y={pos.y:.3f} z={pos.z:.3f} "
              f"Rotation: x={rot.x:.3f} y={rot.y:.3f} z={rot.z:.3f} w={rot.w:.3f}")

    # Print node info once
    if not self.printed_node_info and skeleton.info:
        glove_id = skeleton.info.gloveId
        node_count = skeleton.info.nodesCount

        # Get node info array
        node_info_array = ffi.new("NodeInfo[]", node_count)
        result = lib.CoreSdk_GetRawSkeletonNodeInfoArray(glove_id, node_info_array, node_count)

        if result == SDKReturnCode.Success:
            print(f"\nReceived Skeleton glove data from Core.")
            print(f"Skeletons: {len(self.raw_skeleton)}, First skeleton glove id: {glove_id}")
            print("Printing Node Info:")

            for i in range(node_count):
                node_info = node_info_array[i]
                print(f"  Node {node_info.nodeId}: parent={node_info.parentId}, side={node_info.side}")

            self.printed_node_info = True

Data Structures

ClientRawSkeleton

Container for skeleton data from a single callback:

class ClientRawSkeleton:
    def __init__(self):
        self.info = None        # RawSkeletonInfo (metadata)
        self.nodes = []         # List of SkeletonNode (transforms)

SkeletonNode

Individual skeleton node with transform data:

node.transform.position:
  x, y, z (float)          # Position in 3D space

node.transform.rotation:
  x, y, z, w (float)       # Quaternion rotation (w is scalar)

RawSkeletonInfo

Metadata about a skeleton:

info.gloveId               # Identifier of the glove/hand
info.nodesCount            # Number of nodes in this skeleton
info.publishTime           # Timestamp when data was captured

NodeInfo

Information about a node's role in the skeleton hierarchy:

node_info.nodeId           # Node identifier
node_info.parentId         # Parent node ID (-1 for root)
node_info.side             # Left (1) or Right (2)
node_info.chainType        # Hand, finger, etc.
node_info.fingerJointType  # Joint type (if applicable)

Connection Modes Explained

Integrated Mode

result = lib.CoreSdk_InitializeIntegrated()
# SDK runs fully integrated, no external MANUS Core needed

The integrated mode runs the entire SDK within your application. This is useful when you want complete control over glove management and don't want to depend on MANUS Core running separately.

Local Mode

result = lib.CoreSdk_InitializeCore()
# Connects to localhost

Local mode connects to a MANUS Core instance running on the same machine. This is common for development and testing.

Remote Mode

result = lib.CoreSdk_InitializeCore()
# Searches network for MANUS Core instances and displays host list

Remote mode searches the local network for available MANUS Core instances. After discovering hosts, the user selects which instance to connect to.

Important Notes

Direct CFFI Calls

This example uses direct CFFI calls (lib.CoreSdk_*) rather than wrapper functions. CFFI (C Foreign Function Interface) is used to call C functions from Python. The Python SDK comes with pre-compiled CFFI extensions.

Thread Safety

• Callbacks execute in SDK threads, NOT your main thread
• Use thread-safe primitives (locks, events) when sharing data between callback and main thread
• This example uses skeleton_mutex to protect skeleton data

with self.skeleton_mutex:
    self.next_raw_skeleton = new_skeletons

Connection Retry Logic

The example includes a retry loop for connection failures:

while not self.connect():
    print("Could not connect. Trying again...")
    time.sleep(1.0)

This is important for network-based connections that may fail initially.

Host Discovery Timeout

Different timeouts are used for different modes:

Mode	Timeout	Purpose
Local	1s	Only checks localhost, instant response
Remote	3s	Allows network propagation and discovery

Common Modifications

Increase Update Frequency

# Instead of 30 FPS (33ms)
time.sleep(0.033)

# Use 60 FPS (16ms)
time.sleep(0.016)

# Or 10 FPS for terminal display (100ms)
time.sleep(0.1)

Process Multiple Skeletons

if self.raw_skeleton:
    for skeleton in self.raw_skeleton:
        glove_id = skeleton.info.gloveId
        print(f"Processing glove {glove_id} with {skeleton.info.nodesCount} nodes")
        # Process skeleton...

Add Error Handling

result = lib.CoreSdk_SomeFunction(...)
if result != SDKReturnCode.Success:
    error_name = SDKReturnCode(result).name
    print(f"Error: {error_name}")
    # Handle error appropriately

Extract Specific Nodes

# Find thumb joint nodes
for node_info in node_info_array:
    if node_info.side == Side.Left and node_info.fingerJointType == FingerJointType.Thumb:
        print(f"Found left thumb node: {node_info.nodeId}")

Troubleshooting

"Failed to initialize SDK"

Check that you have:

• Correct Python version (3.9, 3.10, 3.11, or 3.12)
• For Linux: Required system dependencies installed

"Could not connect" (repeating)

Local mode: Ensure MANUS Core is running

• Windows: Check Services or taskbar for MANUS Core
• Verify it's listening on localhost

Remote mode: Check network connectivity

• Firewall may be blocking UDP discovery
• Ensure MANUS Core machine is on same network
• Try Local mode first to verify SDK works

Integrated mode: Should always connect

• Check console output for any error messages
• Try running with administrator privileges

No raw skeleton data appearing

Verify:

1. Gloves are powered on and connected
2. MANUS Core Dashboard shows glove status (when not using Integrated mode)
3. Callbacks are registered before connecting

Callback not firing

Ensure:

1. register_raw_skeleton_callback() was called successfully
2. CoreSdk_ConnectToHost() returned Success
3. Gloves are actively sending data (check MANUS Core Dashboard)
4. Main program doesn't exit immediately after setup

Next Steps

1. Modify for your use case: Adapt the skeleton callback to extract the data you need
2. Add more callbacks: Register ergonomics or landscape callbacks as needed
3. Try SDK Client: See SDK Client Documentation for advanced features
4. Integrate into your project: Copy relevant code sections to your application

See Also

• Getting Started Guide
• SDK Client Documentation
• API SDK Function