Skip to content

License requirement

The functionality described requires a MANUS Bodypack or a MANUS license key with the SDK feature enabled.

SDK Client example

Introduction

The SDK Client example functions as a demonstration of all major SDK functions available through Python. It is a menu-driven console application that shows how to connect to MANUS Core, how to set up the coordinate system, and how to handle multiple data streams including gloves, ergonomics, landscape (device inventory), raw skeleton data, and processed skeleton animations. The example client is written in Python and demonstrates best practices for working with the MANUS SDK in an interactive environment.

Location: Python/examples/sdk_client.py in the MANUS SDK package

The code snippets in this article are pulled directly from examples/sdk_client.py, the source file is noted in each snippet's title.

Features:

  • Interactive menu system for exploring all data streams
  • Glove ergonomics data visualization
  • Haptic feedback control
  • Processed and raw skeleton setup, loading, and visualization
  • Raw device (IMU) sensor data stream
  • Landscape view
  • Glove calibration workflow
  • Glove pairing and unpairing
  • Tracker management
  • User management (add, remove, assign dongles/gloves, reorder)
  • Tracking system timeout and per-system settings

SDK Callbacks

All the data streams MANUS Core provides are structured as callbacks. A callback is a function that you pass to a CoreSdk_RegisterCallback... function. These functions are called on another thread when data becomes available. This way, applications do not need to poll the SDK to determine if there is new data.

It is good practice to register all the callbacks you require after initializing the SDK and before connecting to MANUS Core. You only need to register the callbacks which you intend to use; if you do not plan to use certain data streams, this could reduce network traffic from MANUS Core.

A best practice for data received on another thread is to save the data to a thread-safe location and process it on your own thread. This way, you do not block the callback thread, which could lead to delayed data transfer.

Raw Skeleton Callback

OnRawSkeletonCallback (examples/sdk_client.py)
def on_raw_skeleton_callback(self, stream_info):
    """Callback for raw skeleton data - caches by glove ID since callbacks return one glove at a time"""
    try:
        skeletons_count = stream_info.skeletonsCount
        next_data = {}

        for i in range(skeletons_count):
            # Get skeleton info (metadata such as glove ID and node count)
            info = ffi.new("RawSkeletonInfo*")
            result = lib.CoreSdk_GetRawSkeletonInfo(i, info)
            if result != SDKReturnCode.Success:
                continue

            # Get the skeleton nodes (transforms) for this glove
            nodes_array = ffi.new("SkeletonNode[]", info.nodesCount)
            result = lib.CoreSdk_GetRawSkeletonData(i, nodes_array, info.nodesCount)
            if result != SDKReturnCode.Success:
                continue

            # Get node info array for this glove (hierarchy and chain information)
            node_info_array = ffi.new("NodeInfo[]", info.nodesCount)
            result = lib.CoreSdk_GetRawSkeletonNodeInfoArray(info.gloveId, node_info_array, info.nodesCount)

            # Make deep copies of the data to avoid CFFI memory reuse issues
            skeleton_data = {
                'glove_id': info.gloveId,
                'nodes_count': info.nodesCount,
                'publish_time': info.publishTime,
                'nodes': []
            }

            for j in range(info.nodesCount):
                node = nodes_array[j]
                node_data = {
                    'id': node.id,
                    'position': {
                        'x': node.transform.position.x,
                        'y': node.transform.position.y,
                        'z': node.transform.position.z
                    },
                    'rotation': {
                        'w': node.transform.rotation.w,
                        'x': node.transform.rotation.x,
                        'y': node.transform.rotation.y,
                        'z': node.transform.rotation.z
                    },
                    'scale': {
                        'x': node.transform.scale.x,
                        'y': node.transform.scale.y,
                        'z': node.transform.scale.z
                    }
                }

                # Add node info if available
                if result == SDKReturnCode.Success:
                    node_info = node_info_array[j]
                    node_data['node_info'] = {
                        'node_id': node_info.nodeId,
                        'parent_id': node_info.parentId,
                        'chain_type': node_info.chainType,
                        'side': node_info.side,
                        'finger_joint_type': node_info.fingerJointType
                    }

                skeleton_data['nodes'].append(node_data)

            next_data[info.gloveId] = skeleton_data

        # Hand the data over to the main thread instead of processing it here,
        # blocking the callback thread would delay incoming data
        with self.raw_skeleton_data_mutex:
            self.next_raw_skeleton_data = next_data

    except Exception as e:
        self.log("ERROR", f"Raw skeleton callback error: {e}")
        print(f"Error in raw skeleton callback: {e}")

Ergonomics Callback

OnErgonomicsCallback (examples/sdk_client.py)
def on_ergonomics_callback(self, ergonomics_stream):
    """Callback for ergonomics data"""
    try:
        # Update cached ergonomics data by glove ID
        count = ergonomics_stream.dataCount
        next_ergonomics_data = {}

        for i in range(count):
            data = ergonomics_stream.data[i]
            glove_id = data.id

            # Make a deep copy of the data since CFFI structures may be reused
            data_copy = {
                'id': data.id,
                'isUserID': data.isUserID,
                'data': list(data.data)  # Convert CFFI array to Python list
            }

            next_ergonomics_data[glove_id] = data_copy

        # Hand the data over to the main thread instead of processing it here
        with self.ergonomics_mutex:
            self.next_ergonomics_data = next_ergonomics_data
    except Exception as e:
        self.log("ERROR", f"Ergonomics callback error: {e}")
        print(f"Error in ergonomics callback: {e}")

Landscape Callback

OnLandscapeCallback (examples/sdk_client.py)
def on_landscape_callback(self, landscape):
    """Callback for landscape data"""
    try:
        # Quick validation before storing - check if counts are reasonable
        glove_count = landscape.gloveDevices.gloveCount
        dongle_count = landscape.gloveDevices.dongleCount
        user_count = landscape.users.userCount
        skeleton_count = landscape.skeletons.skeletonCount
        tracker_count = landscape.trackers.trackerCount

        # Only process if counts are valid
        if not (0 <= glove_count <= 32 and 
                0 <= dongle_count <= 32 and 
                0 <= user_count <= 10 and 
                0 <= skeleton_count <= 32 and 
                0 <= tracker_count <= 32):
            return  # Skip invalid data

After this validation, the remainder of the function deep-copies the landscape structure into a Python dictionary so the data stays valid outside the callback.

Raw Device Data Callback

OnRawDeviceDataCallback (examples/sdk_client.py)
def on_raw_device_data_callback(self, raw_device_info):
    """Callback for raw device sensor data"""
    try:
        raw_device_data_count = raw_device_info.rawDeviceDataCount

        # Track callback invocation count
        self.raw_device_callback_count = getattr(self, 'raw_device_callback_count', 0) + 1
        self.last_raw_device_data_count = raw_device_data_count

        next_raw_device_data = {}
        for i in range(raw_device_data_count):
            raw_device = ffi.new("RawDeviceData*")
            result = lib.CoreSdk_GetRawDeviceData(i, raw_device)
            if result != SDKReturnCode.Success:
                continue

            device_id = raw_device.id
            sensor_count = raw_device.sensorCount

            # Deep copy the sensor transforms (position, rotation, scale)
            sensors = []
            for sensor_idx in range(sensor_count):
                sensor_transform = raw_device.sensorData[sensor_idx]
                sensor_data = {
                    'position': {
                        'x': sensor_transform.position.x,
                        'y': sensor_transform.position.y,
                        'z': sensor_transform.position.z
                    },
                    'rotation': {
                        'w': sensor_transform.rotation.w,
                        'x': sensor_transform.rotation.x,
                        'y': sensor_transform.rotation.y,
                        'z': sensor_transform.rotation.z
                    },
                    'scale': {
                        'x': sensor_transform.scale.x,
                        'y': sensor_transform.scale.y,
                        'z': sensor_transform.scale.z
                    }
                }
                sensors.append(sensor_data)

            # Cache device data by device ID
            device_data = {
                'id': device_id,
                'sensor_count': sensor_count,
                'sensors': sensors,
                'rotation': {
                    'w': raw_device.rotation.w,
                    'x': raw_device.rotation.x,
                    'y': raw_device.rotation.y,
                    'z': raw_device.rotation.z
                }
            }

            next_raw_device_data[device_id] = device_data

        # Hand the data over to the main thread instead of processing it here
        with self.raw_device_data_mutex:
            self.next_raw_device_data = next_raw_device_data
    except Exception as e:
        self.log("ERROR", f"Raw device data callback error: {e}")
        print(f"Error in raw device data callback: {e}")

This callback provides access to raw sensor data including accelerometers, gyroscopes, and other motion sensors from connected Manus devices. Each sensor's position and rotation is available for custom processing.

Skeleton Callback

OnSkeletonCallback (examples/sdk_client.py)
def on_skeleton_callback(self, stream_info):
    """Callback for processed skeleton data"""
    try:
        skeletons_count = stream_info.skeletonsCount
        new_skeletons = []

        for i in range(skeletons_count):
            # Get skeleton info (skeleton ID and node count)
            info = ffi.new("SkeletonInfo*")
            result = lib.CoreSdk_GetSkeletonInfo(i, info)
            if result != SDKReturnCode.Success:
                continue

            # Get the skeleton nodes with their animation transforms
            nodes_array = ffi.new("SkeletonNode[]", info.nodesCount)
            result = lib.CoreSdk_GetSkeletonData(i, nodes_array, info.nodesCount)
            if result != SDKReturnCode.Success:
                continue

            # Make deep copies of the data, the CFFI buffers are reused by the SDK
            skeleton_data = {
                'skeleton_id': info.id,
                'nodes_count': info.nodesCount,
                'publish_time': info.publishTime,
                'nodes': []
            }

            for j in range(info.nodesCount):
                node = nodes_array[j]
                node_data = {
                    'id': node.id,
                    'position': {
                        'x': node.transform.position.x,
                        'y': node.transform.position.y,
                        'z': node.transform.position.z
                    },
                    'rotation': {
                        'w': node.transform.rotation.w,
                        'x': node.transform.rotation.x,
                        'y': node.transform.rotation.y,
                        'z': node.transform.rotation.z
                    },
                    'scale': {
                        'x': node.transform.scale.x,
                        'y': node.transform.scale.y,
                        'z': node.transform.scale.z
                    }
                }

                # Add node_info from the skeleton setup metadata if available
                if info.id in self.skeleton_metadata and node.id in self.skeleton_metadata[info.id]:
                    chain_info = self.skeleton_metadata[info.id][node.id]
                    node_data['node_info'] = chain_info

                skeleton_data['nodes'].append(node_data)

            new_skeletons.append(skeleton_data)

        # Hand the data over to the main thread instead of processing it here
        with self.skeleton_mutex:
            self.next_skeletons = new_skeletons
    except Exception as e:
        self.log("ERROR", f"Skeleton callback error: {e}")
        print(f"Error in skeleton callback: {e}")

Tracker Callback

OnTrackerCallback
def on_tracker_callback(self, stream_info):
    try:
        tracker_count = stream_info.trackerCount
        if tracker_count == 0:
            return

        next_tracker_data = {}
        for i in range(tracker_count):
            data = ffi.new("TrackerData*")
            result = lib.CoreSdk_GetTrackerData(i, data)
            if result != SDKReturnCode.Success:
                continue
            tracker_id = ffi.string(data.trackerId.id).decode('utf-8', errors='replace')
            next_tracker_data[tracker_id] = {
                'tracker_type': int(data.trackerType),
                'user_id': int(data.userId),
                'is_hmd': bool(data.isHmd),
                'position': (data.position.x, data.position.y, data.position.z),
                'rotation': (data.rotation.x, data.rotation.y, data.rotation.z, data.rotation.w),
                'quality': int(data.quality),
            }

        with self.tracker_mutex:
            self.next_tracker_data = next_tracker_data

        self.callback_counter['tracker'] += 1
    except Exception as e:
        self.log("ERROR", f"Tracker callback error: {e}")

Client Connection

There are three modes to interact with your glove devices: integrated, local, and remote. The integrated method talks directly to the gloves without needing a MANUS Core instance. The local and remote methods talk to a MANUS Core instance which then communicates with the gloves.

On startup, the choice is given between:

  • Core Integrated mode: The SDK is integrated into the client application.
  • Core Local mode: The SDK will connect to a MANUS Core running locally on this machine.
  • Core Remote mode: The SDK will search and connect to a MANUS Core instance on the network.

When using Remote, the network is scanned, and a list of available MANUS Core instances is returned.

After initialization and connection, the client displays an interactive main menu:

============================================================
MAIN MENU
============================================================
  [G] Gloves & Ergonomics Data
  [S] Skeleton Data
  [R] Raw Skeleton Data
  [D] Raw Device Data (Sensors)
  [L] Landscape Data
  [C] Glove Calibration
  [P] Pairing / Unpairing
  [K] Skeleton Management
  [T] Trackers
  [Y] Tracking Settings
  [U] Users
  [Q] Quit
============================================================
Option Description Hotkey
[G] Display glove & ergonomics data G
[S] Display processed skeleton data S
[R] Display raw skeleton data R
[D] Display raw device sensor data D
[L] Display landscape (system overview) L
[C] Glove calibration menu C
[P] Pairing / Unpairing menu P
[K] Skeleton management (load/unload) K
[T] Trackers menu T
[Y] Tracking settings menu Y
[U] Users management menu U
[Q] Exit the application Q

Retrieving a license

The client can retrieve a dongle's newest license online from the MANUS license service. It is reached from the Pairing / Unpairing menu ([P]), which offers [R] Retrieve first dongle's license.

Retrieving the license and writing back the newest signed license:

Retrieve the first dongle's license
def retrieve_license(self):
    dongles = self.landscape.get('dongles', []) if self.landscape else []
    if not dongles:
        self.log("WARN", "No dongle available to retrieve a license for")
        return

    dongle_id = dongles[0]['id']
    response = ffi.new("Response*")
    result = lib.CoreSdk_RetrieveLicense(dongle_id, response)
    if result == SDKReturnCode.Success:
        msg = ffi.string(response.message.message).decode('utf-8', errors='replace')
        self.log("INFO", f"License retrieved for dongle {dongle_id}: {msg}")
    else:
        self.log("ERROR", f"Retrieve license failed: {SDKReturnCode(result).name}")

Skeleton Management

The client includes skeleton setup and management functionality that allows you to create and load hand skeletons with proper node and chain hierarchies.

Loading a Test Skeleton

To load a hand skeleton:

  1. From the main menu, select [K] for Skeleton Management
  2. Select [L] to load a left-hand skeleton or [R] for right-hand
  3. The skeleton will be created with:
  • 1 root hand node
  • 20 finger joint nodes (5 fingers × 4 joints)
  • 6 chains (1 hand + 5 finger chains)

The skeleton will be fully animated with data from the connected gloves and you'll see the processed skeleton data in the [S] Skeleton Data view.

Unloading a Skeleton

From the Skeleton Management menu, select [U] to unload the currently loaded skeleton.

Helper Functions

_create_node_setup()

Create Node Setup (examples/sdk_client.py)
def _create_node_setup(self, node_id, parent_id, x, y, z, name=""):
    """Helper: create and initialize a NodeSetup cdata value and return it (by-value).

    Args:
        node_id: Unique identifier for this node
        parent_id: ID of the parent node in the hierarchy
        x, y, z: Position coordinates
        name: Optional name for the node (UTF-8, up to 64 bytes)

    Returns:
        NodeSetup cdata value with all fields initialized:
        position at (x, y, z), identity rotation, unit scale,
        type Joint and no special settings (IK, Foot, Leaf, etc.)
    """
    from manus_sdk.generated._enums import NodeType, NodeSettingsFlag

    node = ffi.new("NodeSetup*")
    # Manual zero initialization since _Init functions aren't exported
    node.id = node_id
    node.parentID = parent_id
    node.type = NodeType.Joint
    node.settings.usedSettings = NodeSettingsFlag.None_
    node.transform.position.x = x
    node.transform.position.y = y
    node.transform.position.z = z
    node.transform.rotation.w = 1.0
    node.transform.rotation.x = 0.0
    node.transform.rotation.y = 0.0
    node.transform.rotation.z = 0.0
    node.transform.scale.x = 1.0
    node.transform.scale.y = 1.0
    node.transform.scale.z = 1.0
    if name:
        name_bytes = name.encode('utf-8')
        ffi.memmove(node.name, name_bytes, min(len(name_bytes), 64))
    return node[0]

Initializes a node with:

  • Position, rotation (identity), and scale (1.0)
  • Type set to Joint
  • No special settings (IK, Foot, etc.)
  • Name as UTF-8 string (up to 64 bytes)

_setup_hand_nodes()

Setup Hand Nodes (examples/sdk_client.py)
def _setup_hand_nodes(self, setup_index, side_enum):
    """Create nodes for a simple hand skeleton (root + 5 fingers x 4 joints).

    Args:
        setup_index: Skeleton setup index from CoreSdk_CreateSkeletonSetup
        side_enum: Side.Left or Side.Right

    Returns:
        True if successful, False otherwise

    Node structure:
        Node 0: Hand (root)
        Nodes 1-4: Thumb metacarpal -> distal
        Nodes 5-8: Index metacarpal -> distal
        Nodes 9-12: Middle metacarpal -> distal
        Nodes 13-16: Ring metacarpal -> distal
        Nodes 17-20: Pinky metacarpal -> distal
    """
    # Finger and joint counts
    t_NumFingers = 5
    t_NumJoints = 4

    # Left and right hand node positions
    s_LeftHandPositions = [
        (0.025320, -0.024950, 0.0),
        (0.025320 + 0.032742, -0.024950, 0.0),
        (0.025320 + 0.032742 + 0.028739, -0.024950, 0.0),
        (0.025320 + 0.032742 + 0.028739 + 0.028739, -0.024950, 0.0),

        (0.052904, -0.011181, 0.0),
        (0.052904 + 0.038257, -0.011181, 0.0),
        (0.052904 + 0.038257 + 0.020884, -0.011181, 0.0),
        (0.052904 + 0.038257 + 0.020884 + 0.018759, -0.011181, 0.0),

        (0.051287, 0.0, 0.0),
        (0.051287 + 0.041861, 0.0, 0.0),
        (0.051287 + 0.041861 + 0.024766, 0.0, 0.0),
        (0.051287 + 0.041861 + 0.024766 + 0.019683, 0.0, 0.0),

        (0.049802, 0.011274, 0.0),
        (0.049802 + 0.039736, 0.011274, 0.0),
        (0.049802 + 0.039736 + 0.023564, 0.011274, 0.0),
        (0.049802 + 0.039736 + 0.023564 + 0.019868, 0.011274, 0.0),

        (0.047309, 0.020145, 0.0),
        (0.047309 + 0.033175, 0.020145, 0.0),
        (0.047309 + 0.033175 + 0.018020, 0.020145, 0.0),
        (0.047309 + 0.033175 + 0.018020 + 0.019129, 0.020145, 0.0),
    ]

    s_RightHandPositions = [
        (0.025320, 0.024950, 0.0),
        (0.025320 + 0.032742, 0.024950, 0.0),
        (0.025320 + 0.032742 + 0.028739, 0.024950, 0.0),
        (0.025320 + 0.032742 + 0.028739 + 0.028739, 0.024950, 0.0),

        (0.052904, 0.011181, 0.0),
        (0.052904 + 0.038257, 0.011181, 0.0),
        (0.052904 + 0.038257 + 0.020884, 0.011181, 0.0),
        (0.052904 + 0.038257 + 0.020884 + 0.018759, 0.011181, 0.0),

        (0.051287, 0.0, 0.0),
        (0.051287 + 0.041861, 0.0, 0.0),
        (0.051287 + 0.041861 + 0.024766, 0.0, 0.0),
        (0.051287 + 0.041861 + 0.024766 + 0.019683, 0.0, 0.0),

        (0.049802, -0.011274, 0.0),
        (0.049802 + 0.039736, -0.011274, 0.0),
        (0.049802 + 0.039736 + 0.023564, -0.011274, 0.0),
        (0.049802 + 0.039736 + 0.023564 + 0.019868, -0.011274, 0.0),

        (0.047309, -0.020145, 0.0),
        (0.047309 + 0.033175, -0.020145, 0.0),
        (0.047309 + 0.033175 + 0.018020, -0.020145, 0.0),
        (0.047309 + 0.033175 + 0.018020 + 0.019129, -0.020145, 0.0),
    ]

    from manus_sdk.generated._enums import Side
    fingers = s_LeftHandPositions if side_enum == Side.Left else s_RightHandPositions

    # Add root hand node (ID 0)
    root_node = self._create_node_setup(0, 0, 0.0, 0.0, 0.0, "Hand")
    res = lib.CoreSdk_AddNodeToSkeletonSetup(setup_index, root_node)
    if res != SDKReturnCode.Success:
        self.log("ERROR", f"Failed to add root Hand node to skeleton setup: {res}")
        return False

    # Add finger joints
    finger_id = 0
    for i in range(t_NumFingers):
        parent_id = 0
        for j in range(t_NumJoints):
            idx = i * t_NumJoints + j
            x, y, z = fingers[idx]
            node_id = 1 + finger_id + j
            node = self._create_node_setup(node_id, parent_id, x, y, z, "fingerdigit")
            res = lib.CoreSdk_AddNodeToSkeletonSetup(setup_index, node)
            if res != SDKReturnCode.Success:
                self.log("ERROR", f"Failed to add finger node to skeleton setup: {res}")
                return False
            parent_id = node_id
        finger_id += t_NumJoints
    return True

Creates anatomically accurate node positions based on the C++ SDK sample with proper parent-child relationships.

_setup_hand_chains()

Setup Hand Chains (examples/sdk_client.py)
def _setup_hand_chains(self, setup_index, side_enum):
    """Create hand and finger chains for the skeleton (1 hand + 5 finger chains).

    Args:
        setup_index: Skeleton setup index from CoreSdk_CreateSkeletonSetup
        side_enum: Side.Left or Side.Right

    Returns:
        True if successful, False otherwise

    Chain structure:
        Chain 0: Hand (wrist) - contains all finger chain IDs
        Chain 1: Thumb - nodes 1-4
        Chain 2: Index - nodes 5-8
        Chain 3: Middle - nodes 9-12
        Chain 4: Ring - nodes 13-16
        Chain 5: Pinky - nodes 17-20

    Settings:
        - Hand motion: IMU (Inertial Measurement Unit)
        - All finger chains linked to the hand chain
        - Leaf nodes disabled (chains extend to the finger tips)
    """
    from manus_sdk.generated._enums import ChainType, HandMotion

    # Hand chain settings
    cs = ffi.new("ChainSettings*")
    # Manual initialization - set default values
    cs.usedSettings = ChainType.Hand
    cs.hand.handMotion = HandMotion.IMU
    cs.hand.fingerChainIdsUsed = 5
    cs.hand.fingerChainIds[0] = 1
    cs.hand.fingerChainIds[1] = 2
    cs.hand.fingerChainIds[2] = 3
    cs.hand.fingerChainIds[3] = 4
    cs.hand.fingerChainIds[4] = 5

    chain = ffi.new("ChainSetup*")
    # Manual initialization
    chain.id = 0
    chain.type = ChainType.Hand
    chain.dataType = ChainType.Hand
    chain.side = side_enum
    chain.dataIndex = 0
    chain.nodeIdCount = 1
    chain.nodeIds[0] = 0
    chain.settings = cs[0]

    res = lib.CoreSdk_AddChainToSkeletonSetup(setup_index, chain[0])
    if res != SDKReturnCode.Success:
        self.log("ERROR", f"Failed to add Hand chain to skeleton setup: {res}")
        return False

    # Finger chains
    finger_types = [ChainType.FingerThumb, ChainType.FingerIndex, ChainType.FingerMiddle, ChainType.FingerRing, ChainType.FingerPinky]
    for i in range(5):
        cs2 = ffi.new("ChainSettings*")
        # Manual initialization
        cs2.usedSettings = finger_types[i]
        cs2.finger.handChainId = 0
        cs2.finger.metacarpalBoneId = -1
        cs2.finger.useLeafAtEnd = False

        ch = ffi.new("ChainSetup*")
        # Manual initialization
        ch.id = i + 1
        ch.type = finger_types[i]
        ch.dataType = finger_types[i]
        ch.side = side_enum
        ch.dataIndex = 0
        ch.nodeIdCount = 4
        if i == 0:
            ch.nodeIds[0] = 1
            ch.nodeIds[1] = 2
            ch.nodeIds[2] = 3
            ch.nodeIds[3] = 4
        else:
            ch.nodeIds[0] = (i * 4) + 1
            ch.nodeIds[1] = (i * 4) + 2
            ch.nodeIds[2] = (i * 4) + 3
            ch.nodeIds[3] = (i * 4) + 4
        ch.settings = cs2[0]

        res = lib.CoreSdk_AddChainToSkeletonSetup(setup_index, ch[0])
        if res != SDKReturnCode.Success:
            self.log("ERROR", f"Failed to add finger chain to skeleton setup: {res}")
            return False

    return True

Sets up proper chain hierarchies and links fingers to the hand chain.

Skeleton Data View

The Skeleton Data view displays processed skeleton information (animates with glove input after loading):

Compact View: 

============================================================
SKELETON DATA (Processed)
============================================================
[D] Toggle detailed view  |  [B] Back to main menu
============================================================

Received 1 skeleton(s):

Skeleton #1:
  Skeleton ID: 1
  Node Count: 21
  Total Nodes: 21

Detailed View (toggle with [D]): 

============================================================
SKELETON DATA (Processed)
============================================================
[D] Toggle detailed view  |  [B] Back to main menu
============================================================

Received 1 skeleton(s):

Skeleton #1:
  Skeleton ID: 1
  Total Nodes: 21
  All Nodes:
    Node  0 (Node_0                      ): (  0.000,   0.000,   0.000)
    Node  1 (Node_1                      ): (  0.025,  -0.025,   0.000)
    Node  2 (Node_2                      ): (  0.058,  -0.025,   0.000)
    Node  3 (Node_3                      ): (  0.080,  -0.020,   0.000)
    Node  4 (Node_4                      ): (  0.090,   0.000,   0.000)
    ... [16 more finger joint nodes]
    Node 20 (Node_20                     ): (  0.090,   0.030,   0.000)

Data Stream Views

Raw Skeleton Data

The Raw Skeleton view displays joint positions and rotations for one or both hands:

============================================================
RAW SKELETON DATA - LEFT HAND
============================================================
[H] Toggle hand  |  [D] Toggle view  |  [B] Back
============================================================

Left Hand Skeleton:
  Glove ID: 1234567899 (0x499602D3)
  Node Count: 25
  Total Nodes: 25

Controls:

  • [H]: Switch between left and right hand display
  • [D]: Toggle between compact (hand structure summary) and detailed view (all nodes with transforms)
  • [B]: Return to main menu

Detailed View example:

Raw Skeleton - Detailed View:

Left Hand Skeleton:
  Glove ID: 1234567899 (0x499602D3)
  Node Count: 25
  Total Nodes: 25
  All Nodes:
    Node  0 (L_Hand): (  0.000,   0.000,   0.000)
    Node  1 (L_Thumb_Metacarpal): (  0.025,  -0.025,   0.000)
    Node  2 (L_Thumb_Proximal): (  0.058,  -0.025,   0.000)
    Node  3 (L_Thumb_Intermediate): (  0.080,  -0.020,   0.000)
    Node  4 (L_Thumb_Distal): (  0.090,   0.000,   0.000)
    ... [21 more nodes with positions and rotations]
    Node 24 (L_Pinky_Distal): (  0.090,   0.030,   0.000)

Ergonomics Data

The Ergonomics view displays finger joint bend and spread angles from connected gloves:

============================================================
GLOVES & ERGONOMICS DATA
============================================================
Haptics (HOLD): [1-5] Left (pinky-thumb) [6-0] Right (thumb-pinky)
============================================================

Left Glove:
  Glove ID: 2941338881 (0xAF514501)
  Joint Angles (degrees):
  Thumb : Spread=-36.68°  MCP= 44.20°  PIP=  0.86°  DIP= 30.91°
  Index : Spread= 50.52°  MCP=-76.32°  PIP= 53.92°  DIP=113.67°
  Middle: Spread= 49.25°  MCP=-49.85°  PIP= 63.82°  DIP= 89.94°
  Ring  : Spread= 10.89°  MCP=-30.96°  PIP=  1.79°  DIP= 60.08°
  Pinky : Spread= 10.44°  MCP=-25.79°  PIP=  0.62°  DIP= 15.35°

Right Glove:
  Glove ID: 2941338882 (0xAF514502)
  Joint Angles (degrees):
  Thumb : Spread= 35.12°  MCP= 42.85°  PIP=  1.23°  DIP= 32.15°
  Index : Spread= 48.67°  MCP=-75.45°  PIP= 55.33°  DIP=115.22°
  Middle: Spread= 47.89°  MCP=-48.92°  PIP= 64.15°  DIP= 91.28°
  Ring  : Spread= 12.34°  MCP=-32.11°  PIP=  2.56°  DIP= 61.45°
  Pinky : Spread=  9.78°  MCP=-26.34°  PIP=  0.89°  DIP= 16.72°

Data Explanation:

  • Glove ID: Hardware identifier (shown in decimal and hexadecimal)
  • Spread: Finger abduction/adduction angle from palm center (negative = towards palm)
  • MCP: Metacarpophalangeal joint bend (knuckle) in degrees
  • PIP: Proximal interphalangeal joint bend (middle joint) in degrees
  • DIP: Distal interphalangeal joint bend (tip joint) in degrees

All angles are in degrees with real-time updates at ~100 Hz from connected gloves.

Controls:

  • [B]: Return to main menu

Landscape Data

The Landscape view displays the system overview showing all connected devices:

============================================================
LANDSCAPE DATA - System Overview (COMPACT)
============================================================
[D] Toggle detailed view  |  [B] Back to main menu
============================================================

╔═══ Landscape ═══
║
╠═ Devices
║  ├─ Dongles: 2
║  │  • ID:0x39BFB011 FW:5.16.0
║  │  • ID:0x39BFBFFF FW:5.16.0
║  ├─ Gloves: 2
║  │  • ID:1 (Left) - Dongle:0
║  │  • ID:2 (Right) - Dongle:1
║  
║
╠─ Users: 1
║  • WiredUser_0x0000000 (ID:0) Gloves:[L-R]
║
╠─ Skeletons: 1
║  • Skeleton 1 (Hand) User:0
║
╠═ Trackers: 0
║
└─ Settings
   • Manus Core: v3.1.1
   • Mode: Live
   • Max Glove Pairs: 2
   • Features: SDK, Recording, Exporting

Compact View provides:

  • Dongle count and firmware versions
  • Connected glove list with IDs and assignments
  • Tracker inventory
  • User count
  • Skeleton count

Detailed View expands to show all device fields and properties for deep inspection.

Controls:

  • [D]: Toggle to detailed view (shows all fields)
  • [B]: Return to main menu

Raw Device Data

The Raw Device Data view displays sensor information from connected Manus devices (IMUs, accelerometers, gyroscopes, etc.):

============================================================
RAW DEVICE DATA - Sensor Information
============================================================
[B] Back to main menu
============================================================

Connected Devices: 2

Device ID: 0x39BFB011
  Sensor Count: 2
  Sensors:
    Sensor 1:
      Position: (  0.000,   0.000,   0.000)
      Rotation: (W: 1.000, X: 0.000, Y: 0.000, Z: 0.000)
    Sensor 2:
      Position: (  0.050,   0.000,   0.000)
      Rotation: (W: 0.999, X: 0.045, Y: 0.000, Z: 0.000)
  Device Rotation: (W: 0.998, X:-0.063, Y: 0.000, Z: 0.000)

Device ID: 0x39BFBFFF
  Sensor Count: 2
  Sensors:
    Sensor 1:
      Position: (  0.000,   0.000,   0.000)
      Rotation: (W: 1.000, X: 0.000, Y: 0.000, Z: 0.000)
    Sensor 2:
      Position: (  0.050,   0.000,   0.000)
      Rotation: (W: 0.998, X:-0.063, Y: 0.000, Z: 0.000)
  Device Rotation: (W: 0.999, X: 0.031, Y: 0.000, Z: 0.000)

Data Displayed:

  • Device ID: Hardware identifier for each device
  • Sensor Count: Number of motion sensors in the device
  • Sensor Position/Rotation: Transform of each sensor relative to device origin
  • Device Rotation: Overall device orientation

Use Cases:

  • Direct access to raw IMU data for custom motion processing
  • Sensor fusion and filtering
  • Advanced hand tracking algorithms
  • Motion capture and animation

Controls:

  • [B]: Return to main menu

Trackers

The Trackers menu provides live visualization of all connected trackers and tools for assigning them to users.

============================================================
TRACKERS
============================================================
[O] Toggle Test Tracker (OFF)  [G] Toggle View (Global)
[A] Assign Tracker  [U] Unassign Tracker  [T] Set Offset
[B] Back to main menu
============================================================

Controls

Option Description
[O] Toggle a virtual test tracker sent to MANUS Core
[G] Switch between Global view (all trackers) and Per-User view
[A] Assign the first available tracker to the first user with a Left Hand role
[U] Unassign the first assigned tracker (sets its user ID to 0)
[T] Set a tracker offset for the first user's left-hand tracker
[B] Return to the main menu

Test Tracker

Toggling the test tracker ([O]) sends a virtual TrackerData to MANUS Core every display cycle. This is useful for testing tracker integration without physical hardware. The test tracker is a Head-type tracker fixed at position (0, 1, 0) with an identity rotation.

Tracker Views

  • Global view: Lists every discovered tracker with its ID, type, position, rotation, and tracking quality.
  • Per-User view: Groups trackers by the user ID they are currently assigned to.

Users

The Users menu provides tools for managing MANUS Core user accounts and their device assignments.

============================================================
USERS
============================================================
[Z] Disable Auto-Assignment  [X] Enable Auto-Assignment
[A] Add User   [R] Remove User
[D] Assign Dongle  [U] Unassign Dongle
[G] Assign Glove   [W] Unassign Glove
[I] Move User Up   [K] Move User Down
[N] Change Username
[B] Back to main menu
============================================================

Controls

Option Description
[Z] Disable automatic glove-to-user assignment
[X] Enable automatic glove-to-user assignment
[A] Create a new user
[R] Remove the last user in the list
[D] Assign the first available unassigned dongle to a user
[U] Unassign the dongle from the first user that has one assigned
[G] Assign the first available unpaired glove to a user matching the glove's side
[W] Unassign the first assigned glove from its user
[I] Move the user with the highest ID one position up
[K] Move the user with the highest ID one position down
[N] Append " but different" to the first user's name (demonstrates CoreSdk_SetUserName)
[B] Return to the main menu

Tracking Settings

The Tracking Settings menu exposes controls for tracker timeout behaviour and per-system configuration.

============================================================
TRACKING SETTINGS
============================================================
[T] Toggle Tracker Timeouts
[L] Cycle Timeout Duration (10s / 30s)
[0-5] Open Tracker System  |  [B] Back
============================================================

Tracker Timeouts: Enabled  (Duration: 30.0s)

Controls

Option Description
[T] Toggle tracker timeouts on or off
[L] Cycle the timeout duration between 10 s and 30 s
[0–5] Open the submenu for the corresponding tracker system
[B] Return to the main menu

Tracker System Submenu

Selecting a numbered tracker system opens its settings submenu:

============================================================
TRACKER SYSTEM
============================================================
[A] Toggle Active
[0-9] Select Setting
[B] Back to tracking settings
============================================================

Each tracker system can be enabled or disabled with [A]. Its settings list contains typed values (integer, boolean, IP address, or file path). Select a setting by number to edit it interactively.

Architecture

Skeleton Setup Flow

The skeleton loading process follows this sequence:

  1. Create Skeleton Setup → CoreSdk_CreateSkeletonSetup() returns setup index
  2. Add Hand Nodes → _setup_hand_nodes() adds 21 nodes with positions and hierarchy
  3. Add Hand Chains → _setup_hand_chains() creates chain topology and links
  4. Load Skeleton → CoreSdk_LoadSkeleton() returns skeleton ID for activation
  5. Build Metadata → Extract chains to build node→chain_type mapping
  6. Start Streaming → Skeleton is now animated and generates callbacks

Data Caching

All callback data is cached by glove ID or converted to Python dictionaries to prevent CFFI memory reuse issues:

Data Caching
# Raw skeleton cached by glove ID
self.raw_skeleton_data = {
    1: {'nodes_count': 43, 'nodes': [...]},  # Left glove
    2: {'nodes_count': 43, 'nodes': [...]}   # Right glove
}

# Ergonomics cached by glove ID
self.ergonomics_data = {
    1: {'id': 1, 'data': [...]},
    2: {'id': 2, 'data': [...]}
}

# Landscape is a deep copy of entire structure
self.landscape_data = {
    'dongles': [...],
    'gloves': [...],
    'users': [...],
    ...
}

This caching strategy ensures data persists even when CFFI structures are reused by the SDK.

Thread Safety

In the SDK_Client, all callback data is protected using a double-buffer pattern: callbacks write to a next_* staging variable under a mutex, and the main thread swaps the staged data into the live variable in update_before_displaying_data(). This avoids blocking the callback thread during rendering.

Thread-Safe Double-Buffer
import threading

self.raw_skeleton_data_mutex = threading.Lock()

# In callback (SDK thread):
with self.raw_skeleton_data_mutex:
    self.next_raw_skeleton_data = next_data  # stage new data

# In main thread (update_before_displaying_data):
with self.raw_skeleton_data_mutex:
    if self.next_raw_skeleton_data is not None:
        self.raw_skeleton_data.update(self.next_raw_skeleton_data)
        self.next_raw_skeleton_data = None  # consume

Performance Considerations

Aspect Implementation Notes
Frame Rate 10 FPS limit Reduces terminal flicker glove data rate is much higher
Data Copying Deep CFFI→dict Prevents memory reuse issues, uses more memory
Thread Safety Mutex per data stream Callbacks run in SDK threads
Memory Dictionary caching Requires more memory than direct CFFI use

Data Structures

ClientRawSkeleton

Container for skeleton callback data:

skeleton_data = {
    'nodes_count': 43,
    'side': 1,  # 1=Left, 2=Right
    'nodes': [
        {
            'transform': {
                'position': {'x': 0.0, 'y': 0.05, 'z': -0.005},
                'rotation': {'x': 0.0, 'y': 0.087, 'z': -0.005, 'w': 0.996}
            }
        },
        # ... more nodes
    ]
}

Ergonomics Data

ergonomics_data = {
    'id': 1,
    'isUserID': False,
    'data': [12, 45, 32, 28, ...]
}

The data array contains:

  • Spread, MCP, PIP, DIP for each finger (5 fingers × 4 values = 20 values)
  • Additional sensor data and metrics (remaining ~99 values)

Landscape Data

landscape_data = {
    'dongles': [
        {'id': 0x39BFB011, 'firmware': '5.16.0', ...}
    ],
    'gloves': [
        {'id': 1, 'side': 1, 'dongle_id': 0, ...},
        {'id': 2, 'side': 2, 'dongle_id': 1, ...}
    ],
    'users': [],
    'skeletons': [
        {'id': 1, ...},
        {'id': 2, ...}
    ],
    'settings': {...}
}

Customization Examples

Change Frame Rate

# In __init__:
self.min_display_interval = 0.05  # 20 FPS
# or
self.min_display_interval = 0.033  # 30 FPS

Export Data to File

import json

def export_snapshot(self):
    """Export current data snapshot"""
    snapshot = {
        'timestamp': time.time(),
        'skeleton': self.raw_skeleton_data,
        'ergonomics': self.ergonomics_data,
        'landscape': self.landscape_data
    }

    with open('sdk_data.json', 'w') as f:
        json.dump(snapshot, f, indent=2)

    print("Data exported to sdk_data.json")

Connection Flow

The client initializes in the following order:

  1. Prompt user for connection mode (Integrated/Local/Remote)
  2. Initialize SDK (CoreSdk_InitializeIntegrated or CoreSdk_InitializeCore)
  3. Register all callbacks
  4. Set coordinate system
  5. Search for hosts and establish connection
  6. Enter main menu

If connection fails in Local or Remote modes, the client retries every second until successful.

Troubleshooting

"Could not connect" Loop

  • Local mode: Ensure MANUS Core is running (localhost)
  • Remote mode: Check network connectivity and firewall
  • Integrated mode: Should always work; check console for errors

Data Not Updating

  • Ensure gloves are powered on and connected
  • Verify MANUS Core Dashboard shows glove status
  • Check that callbacks are registered before connecting
  • Verify gloves are actively sending data

Terminal Display Issues

  • Adjust min_display_interval if display updates too frequently/slowly
  • If terminal appears garbled, use full detailed view instead of compact
  • Try reducing other terminal output during operation

Remote Mode Takes Long Time

  • Network discovery takes ~3 seconds by design
  • If no hosts found after 3 seconds, check:
  • MANUS Core running on remote machine
  • Network connectivity between machines

Skeleton Not Appearing in Skeleton Data

  • Ensure you've loaded a skeleton using [K] → [L] or [R]
  • Wait a moment for the first skeleton callback to arrive
  • Verify raw skeleton data is available first (indicates glove connectivity)

API Reference

load_test_skeleton(side)

Loads a complete hand skeleton with proper node and chain hierarchies.

Load Test Skeleton (examples/sdk_client.py)
def load_test_skeleton(self, side):
    """Load a test skeleton for the specified side (left or right).

    Creates a fully configured hand skeleton matching the C++ SDK sample:
    - 1 root hand node
    - 20 finger joint nodes (5 fingers x 4 joints)
    - 6 chains (1 hand chain + 5 finger chains)

    Nodes and chains are created before loading the skeleton so callbacks
    will see a fully populated hand skeleton.

    Args:
        side: 'left' or 'right' to specify which hand to load

    After loading, the skeleton will:
    - Generate skeleton callbacks with animation data
    - Be visible in the Skeleton Data view
    - Be animated by connected glove data
    - Keep streaming until unload_test_skeleton() is called
    """
    try:
        from manus_sdk.generated._enums import SkeletonType, SkeletonTargetType, Side

        # Determine which side
        side_enum = Side.Left if side == 'left' else Side.Right
        side_name = "Left" if side == 'left' else "Right"

        # Create skeleton setup
        setup = ffi.new("SkeletonSetupInfo*")
        # Manual initialization - zero out and set defaults
        setup.type = SkeletonType.Hand
        setup.settings.scaleToTarget = True
        setup.settings.useEndPointApproximations = True
        setup.settings.targetType = SkeletonTargetType.UserIndexData
        setup.settings.skeletonTargetUserIndexData.userIndex = 0            # Set skeleton name
        name = f"{side_name}Hand"
        name_bytes = name.encode('utf-8')
        ffi.memmove(setup.name, name_bytes, min(len(name_bytes), 64))

        # Create skeleton setup
        setup_index_out = ffi.new("uint32_t*")
        result = lib.CoreSdk_CreateSkeletonSetup(setup[0], setup_index_out)

        if result != SDKReturnCode.Success:
            self.log("ERROR", f"Failed to create skeleton setup: {result}")
            return

        setup_index = setup_index_out[0]
        self.temporary_skeletons.append(setup_index)

        # Add nodes and chains for a hand skeleton
        # Note: functions return False on failure and will cleanup the temporary list below
        from manus_sdk.generated._enums import Side as _SideEnum
        if not self._setup_hand_nodes(setup_index, side_enum):
            if setup_index in self.temporary_skeletons:
                self.temporary_skeletons.remove(setup_index)
            return

        if not self._setup_hand_chains(setup_index, side_enum):
            if setup_index in self.temporary_skeletons:
                self.temporary_skeletons.remove(setup_index)
            return

        # Load the skeleton
        skeleton_id_out = ffi.new("uint32_t*")
        result = lib.CoreSdk_LoadSkeleton(setup_index, skeleton_id_out)

        if result != SDKReturnCode.Success:
            self.log("ERROR", f"Failed to load skeleton: {result}")
            if setup_index in self.temporary_skeletons:
                self.temporary_skeletons.remove(setup_index)
            return

        skeleton_id = skeleton_id_out[0]
        if skeleton_id == 0:
            self.log("ERROR", "Failed to assign skeleton ID")
            if setup_index in self.temporary_skeletons:
                self.temporary_skeletons.remove(setup_index)
            return

        # Build skeleton metadata by reading chains from setup
        try:
            from manus_sdk.generated._enums import ChainType

            # Get the number of chains in the setup
            setup_sizes = ffi.new("SkeletonSetupArraySizes*")
            result = lib.CoreSdk_GetSkeletonSetupArraySizes(setup_index, setup_sizes)

            if result == SDKReturnCode.Success and setup_sizes.chainsCount > 0:
                # Allocate array for chains
                chains_array = ffi.new("ChainSetup[]", setup_sizes.chainsCount)
                result = lib.CoreSdk_GetSkeletonSetupChainsArray(setup_index, chains_array, setup_sizes.chainsCount)

                if result == SDKReturnCode.Success:
                    # Build node -> chain_type mapping
                    node_to_chain = {}
                    for chain_idx in range(setup_sizes.chainsCount):
                        chain = chains_array[chain_idx]
                        chain_type = chain.type
                        side = chain.side

                        # Add each node in this chain to the mapping
                        for node_idx in range(chain.nodeIdCount):
                            node_id = chain.nodeIds[node_idx]
                            node_to_chain[node_id] = {
                                'chain_type': chain_type,
                                'side': side,
                                'node_id': node_id,
                                'parent_id': 0,  # We'd need to look this up from NodeSetup if needed
                                'finger_joint_type': 0  # Default, would need to look up if needed
                            }

                    # Store the metadata for this skeleton
                    self.skeleton_metadata[skeleton_id] = node_to_chain
        except Exception as e:
            self.log("WARN", f"Could not build skeleton metadata: {e}")

        self.loaded_skeletons.append(skeleton_id)
        self.log("INFO", f"Loaded {side_name} hand skeleton (ID: {skeleton_id})")

    except Exception as e:
        self.log("ERROR", f"Error loading skeleton: {e}")

Return Values:

  • Logs "Loaded [Side] hand skeleton (ID: X)" on success
  • Logs error message on failure with specific SDK return code

Common Issues:

  • Failed to create skeleton setup: SDK initialization problem
  • Failed to add [object] to skeleton setup: Corrupted node/chain data
  • Failed to load skeleton: Setup data invalid or SDK issue
  • Failed to assign skeleton ID: Serious SDK state issue

unload_test_skeleton()

Unloads the currently loaded test skeleton.

Unload Test Skeleton (examples/sdk_client.py)
def unload_test_skeleton(self):
    """Unload the first loaded skeleton from the system.

    Removes the skeleton from the animation pipeline and cleans up:
    - Skeleton data streaming
    - Internal metadata cache
    - Resources allocated by the SDK

    After calling this, no more skeleton callbacks are received for this
    skeleton and the Skeleton Data view will be empty.
    """
    if not self.loaded_skeletons:
        self.log("WARN", "No loaded skeleton to unload")
        return

    try:
        skeleton_id = self.loaded_skeletons[0]
        result = lib.CoreSdk_UnloadSkeleton(skeleton_id)

        if result == SDKReturnCode.Success:
            self.loaded_skeletons.pop(0)
            # Clean up metadata
            if skeleton_id in self.skeleton_metadata:
                del self.skeleton_metadata[skeleton_id]
            self.log("INFO", f"Unloaded skeleton (ID: {skeleton_id})")
        else:
            self.log("ERROR", f"Failed to unload skeleton: {result}")
    except Exception as e:
        self.log("ERROR", f"Error unloading skeleton: {e}")

Return Values:

  • Logs "Unloaded skeleton (ID: X)" on success
  • Logs "No loaded skeleton to unload" if none are loaded
  • Logs error message on SDK failure

Internal Helper Functions

These are used internally by load_test_skeleton and are documented for reference:

_create_node_setup(node_id, parent_id, x, y, z, name="")

Creates a properly initialized NodeSetup structure with anatomically correct defaults:

  • Position at (x, y, z)
  • Identity rotation (w=1.0, x/y/z=0.0)
  • Unit scale (1.0, 1.0, 1.0)
  • Type: Joint
  • No special settings (IK, Foot, Leaf, etc.)

_setup_hand_nodes(setup_index, side_enum)

Adds 21 nodes to a skeleton setup:

  • 1 root hand node at origin
  • 5 fingers with 4 joints each
  • Positions based on average hand dimensions from C++ sample
  • Proper parent-child hierarchy

_setup_hand_chains(setup_index, side_enum)

Creates 6 chains linking the nodes:

  • Hand chain (wrist) containing all finger chain IDs
  • 5 finger chains with proper joint sequencing
  • Hand motion set to IMU (Inertial Measurement Unit)
  • All fingers linked to hand chain

See Also