Skip to content

License requirement

The functionality described requires a MANUS Bodypack or a MANUS license dongle containing one of these licenses to be connected:
Core Pro Core XR Core Xsens Pro Core Qualisys Pro Core OptiTrack Pro Demo

SDK Minimal Client

Introduction

The SDK Minimal client is an example client that demonstrates the basic functionality of the MANUS SDK. It covers the following steps:

  1. Initialization:

    • The SDK needs to be initialized using the InitializeSDK() function. This sets up the session type and registers the necessary callbacks.
    • The coordinate system for the client is set using the CoreSdk_InitializeCoordinateSystemWithVUH() function.
    • The SDK is initialized but not yet connected to MANUS Core.

  2. Connection:

    • The client attempts to connect locally by calling the ConnectLocally() function in a loop until successful.
    • Once connected, the client sets up the skeleton and starts the main loop.
    • In the main loop, it checks for new skeleton data, signals if there is new data, and waits for the space key to exit.

  3. Setup of a skeleton:

    • The LoadTestSkeleton() function sets up a minimalistic hand skeleton.
    • It creates a SkeletonSetupInfo structure to contain the relevant data.
    • The skeleton is referenced to a user by setting settings.targetype to SkeletonTarget_UserIndexData.
    • The hand nodes and chain nodes are set up using the SetupHandNodes() and SetupHandChains() functions.
    • The skeleton setup is loaded into MANUS Core using the CoreSdk_LoadSkeleton() function.

  4. Skeleton updates:

    • The client waits for new skeleton data via the OnSkeletonStreamCallback() function.
    • The data is copied inside a mutex to ensure thread safety.
    • In this example, the data is only used to post a console output message.

The SDK Minimal client is composed of the manusSDK.dll and the SDKMinimalClient.cpp files. The other files are support files from the SDK and are not important for understanding the basic functions used in this example.

Info

The SDK Minimal client can also be compiled for use on Linux, please refer to the specific guide for compiling on Linux.

Initialize

Before using any functionality of the SDK, it is necessary to initialize it. This ensures that the system is set up correctly and ready for use.

The PlatformSpecificInitialization() function is an optional initializer that prepares console output, but it is not required in a non-console environment.

During the InitializeSDK() function, the SDK is initialized and sets up the type of session it will use to connect to MANUS Core. Typically, this will be of type SessionType_CoreSDK.

Next, the necessary callbacks are registered using the RegisterAllCallbacks() function. It is important to note that there may be additional callbacks beyond the SkeletonStream callback, but they are not included in this example. For more information on other callbacks, please refer to the SDK Client article.

After registering the callbacks, the coordinate system used by the client is set using the CoreSdk_InitializeCoordinateSystemWithVUH() function. In this example, the Unity coordinate system is adopted. It is recommended to align the coordinate system with the application being developed, as this will make it easier to work with the data by letting Core handle the conversion.

The unit scale is specified as a float value. A scale of 1 represents meters, 0.01 represents centimeters, and 0.001 represents millimeters.

The second parameter of the CoreSdk_InitializeCoordinateSystemWithVUH() function indicates whether the coordinates should be set as world coordinates or relative coordinates. In this case, it is set to false to use relative coordinates.

Once the initialization is complete, the SDK is ready to be used, but it is not yet connected to an instance of MANUS Core.

Connection

When running the example it follows the following command structure:

  1. Connect locally, if not successful, just wait and retry
  2. Setup a skeleton
  3. Start the main loop
  4. Check if there is new skeleton data
  5. If there is, signal that there is
  6. Wait for the space key to exit
Run
/// @brief main loop
void SDKMinimalClient::Run()
{
    // first loop until we get a connection locally
    std::cout << "minimal client is connecting to local host. (make sure it is running)\n";
    while (ConnectLocally() != ClientReturnCode::ClientReturnCode_Success)
    {
        // not yet connected. wait
        std::cout << "minimal client could not connect.trying again in a second.\n";
        std::this_thread::sleep_for(std::chrono::milliseconds(1000));
    }
    std::cout << "minimal client is connected, setting up skeletons.\n";
    // then upload a simple skeleton with a chain. this will just be a left hand for the first userindex.
    LoadTestSkeleton();

    // then loop and get its data while waiting for escape key to end it
    while (m_Running)
    {
        // check if there is new data. otherwise we just wait.
        m_SkeletonMutex.lock();
        if (m_NextSkeleton != nullptr)
        {
            if (m_Skeleton != nullptr)delete m_Skeleton;
            m_Skeleton = m_NextSkeleton;
            m_NextSkeleton = nullptr;
        }
        m_SkeletonMutex.unlock();

        if (m_Skeleton != nullptr && m_Skeleton->skeletons.size() != 0)
        {
            // print update
            std::cout << "skeleton data obtained for frame: " << std::to_string(m_FrameCounter) << ".\n";
            m_FrameCounter++;
        }

        std::this_thread::sleep_for(std::chrono::milliseconds(33)); // or roughly 30fps, but good enough to show the results.

        if (GetKeyDown(' ')) // press space to exit
        {
            m_Running = false;
        }
    }
    // then exit.
}

For the connection, the SDK Minimal client currently only tries to connect locally to MANUS Core instances on the machine running the client.

The ConnectLocally() function performs the following steps to set up a connection:

  • First, it calls CoreSdk_LookForHosts() for 1 second, searching only for MANUS Cores on the local pc. If no hosts are found, it returns a failure and stops. The second parameter can be set to false to search for all hosts on the network.

  • Then, it retrieves the number of local hosts found and their host names. If the retrieval fails (e.g., due to no hosts being found), it returns a failure and stops.

  • Finally, it connects to the first available local host. If there are any network settings blocking the connection or if the host is stopped, the connection may fail. The function returns the result of the connection attempt.

Setup a skeleton

The following steps are taken for setting up a hand skeleton and ensuring correct animation:

  1. Setup a container for the skeleton:

    • Create a SkeletonSetupInfo structure.
    • Set the settings.targetype to SkeletonTarget_UserIndexData to reference a user. More info on target types.
    • Set the settings.skeletonTargetUserIndexData.usedIndex to select the user.
    • Set the settings.skeletonType to SkeletonType_Hand for a hand skeleton. More info on skeleton types.

  2. Setup the hand nodes for the skeleton:

    • Define the number of fingers per hand and the number of joints per finger (make sure to include the tip bones).
    • Add the nodes to the skeleton setup using CoreSdk_AddNodeToSkeletonSetup.

  3. Setup the chain nodes for the skeleton:

    • Add the hand chain as the root chain and set its motion to be controlled by the glove wrist IMU.
    • Add the finger chains to describe the fingers.

  4. Load the skeleton into MANUS Core:

    • Use CoreSdk_LoadSkeleton() to send the skeleton data to MANUS Core for animation.

These steps ensure that the hand skeleton is properly set up and animated.

Load Test Skeleton
/// @brief This function sets up a very minimalistic hand skeleton.
/// In order to have any 3d positional/rotational information from the gloves or body,
/// one needs to setup a skeleton on which this data can be applied.
/// In the case of this sample we create a Hand skeleton in order to get skeleton information
/// in the OnSkeletonStreamCallback function. This sample does not contain any 3D rendering, so
/// we will not be applying the returned data on anything.
void SDKMinimalClient::LoadTestSkeleton()
{
    uint32_t t_SklIndex = 0;

    SkeletonSetupInfo t_SKL;
    SkeletonSetupInfo_Init(&t_SKL);
    t_SKL.type = SkeletonType::SkeletonType_Hand;
    t_SKL.settings.scaleToTarget = true;
    t_SKL.settings.targetType = SkeletonTargetType::SkeletonTargetType_UserIndexData;
    //If the glove does not exist then the added skeleton will not be animated.
    //Same goes for any other skeleton made for invalid users/gloves.
    t_SKL.settings.skeletonTargetUserIndexData.userIndex = 0; // just take the first index. make sure this matches in the landscape. 

    CopyString(t_SKL.name, sizeof(t_SKL.name), std::string("LeftHand"));

    SDKReturnCode t_Res = CoreSdk_CreateSkeletonSetup(t_SKL, &t_SklIndex);
    if (t_Res != SDKReturnCode::SDKReturnCode_Success)
    {
        return;
    }

    // setup nodes and chains for the skeleton hand
    if (!SetupHandNodes(t_SklIndex)) return;
    if (!SetupHandChains(t_SklIndex)) return;

    // load skeleton 
    uint32_t t_ID = 0;
    t_Res = CoreSdk_LoadSkeleton(t_SklIndex, &t_ID);
    if (t_Res != SDKReturnCode::SDKReturnCode_Success)
    {
        return;
    }
}

To set up the skeleton, we need to define a container for the skeleton called SkeletonSetupInfo. This container will hold all the relevant data for the skeleton.

In this case, we are setting up a hand skeleton. To do this, we set the settings.skeletonType field of the SkeletonSetupInfo structure to SkeletonType_Hand. This indicates that we are creating a hand skeleton.

Next, we need to specify the target user for the skeleton. We set settings.targetType to SkeletonTarget_UserIndexData, which means that the skeleton will be associated with a specific user based on their index in MANUS Core. In this example, we set settings.skeletonTargetUserIndexData.usedIndex to 0, which means that the first user will be used.

The hand nodes for the skeleton represent the different parts of the hand. These nodes are defined as raw data with relative positions. In this example, we are not including the metacarpals.

The used vectors represent the positions of the hand nodes based on a hand lying flat on a surface, in this example the following nodes are defined:

  • Node 0: Wrist
  • Node 1: Thumb Proximal
  • Node 2: Thumb Intermediate
  • Node 3: Thumb Distal
  • Node 4: Thumb Tip
  • Node 5: Index Proximal
  • Node 6: Index Intermediate
  • Node 7: Index Distal
  • Node 8: Index Tip
  • Node 9: Middle Proximal
  • Node 10: Middle Intermediate
  • Node 11: Middle Distal
  • Node 12: Middle Tip
  • Node 13: Ring Proximal
  • Node 14: Ring Intermediate
  • Node 15: Ring Distal
  • Node 16: Ring Tip
  • Node 17: Pinky Proximal
  • Node 18: Pinky Intermediate
  • Node 19: Pinky Distal
  • Node 20: Pinky Tip

These nodes are added to the skeleton setup using the CoreSdk_AddNodeToSkeletonSetup function.

Once the skeleton nodes are set up, we can proceed to setting up the skeleton chains, which describe the connections between the nodes and drives how they should be animated.

Setup Hand Nodes
/// @brief This support function sets up the nodes for the skeleton hand
/// In order to have any 3d positional/rotational information from the gloves or body,
/// one needs to setup the skeleton on which this data should be applied.
/// In the case of this sample we create a Hand skeleton for which we want to get the calculated result.
/// The ID's for the nodes set here are the same IDs which are used in the OnSkeletonStreamCallback,
/// this allows us to create the link between Manus Core's data and the data we enter here.
bool SDKMinimalClient::SetupHandNodes(uint32_t p_SklIndex)
{
    // Define number of fingers per hand and number of joints per finger
    const uint32_t t_NumFingers = 5;
    const uint32_t t_NumJoints = 4;

    // Create an array with the initial position of each hand node. 
    // Note, these values are just an example of node positions and refer to the hand laying on a flat surface.
    ManusVec3 t_Fingers[t_NumFingers * t_NumJoints] = {
        CreateManusVec3(0.024950f, 0.000000f, 0.025320f), //Thumb CMC joint
        CreateManusVec3(0.000000f, 0.000000f, 0.032742f), //Thumb MCP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.028739f), //Thumb IP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.028739f), //Thumb Tip joint

        //CreateManusVec3(0.011181f, 0.031696f, 0.000000f), //Index CMC joint // Note: we are not adding the matacarpal bones in this example, if you want to animate the metacarpals add each of them to the corresponding finger chain.
        CreateManusVec3(0.011181f, 0.000000f, 0.052904f), //Index MCP joint, if metacarpal is present: CreateManusVec3(0.000000f, 0.000000f, 0.052904f)
        CreateManusVec3(0.000000f, 0.000000f, 0.038257f), //Index PIP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.020884f), //Index DIP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.018759f), //Index Tip joint

        //CreateManusVec3(0.000000f, 0.033452f, 0.000000f), //Middle CMC joint
        CreateManusVec3(0.000000f, 0.000000f, 0.051287f), //Middle MCP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.041861f), //Middle PIP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.024766f), //Middle DIP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.019683f), //Middle Tip joint

        //CreateManusVec3(-0.011274f, 0.031696f, 0.000000f), //Ring CMC joint
        CreateManusVec3(-0.011274f, 0.000000f, 0.049802f),  //Ring MCP joint, if metacarpal is present: CreateManusVec3(0.000000f, 0.000000f, 0.049802f),
        CreateManusVec3(0.000000f, 0.000000f, 0.039736f),  //Ring PIP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.023564f),  //Ring DIP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.019868f),  //Ring Tip joint

        //CreateManusVec3(-0.020145f, 0.027538f, 0.000000f), //Pinky CMC joint
        CreateManusVec3(-0.020145f, 0.000000f, 0.047309f),  //Pinky MCP joint, if metacarpal is present: CreateManusVec3(0.000000f, 0.000000f, 0.047309f),
        CreateManusVec3(0.000000f, 0.000000f, 0.033175f),  //Pinky PIP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.018020f),  //Pinky DIP joint
        CreateManusVec3(0.000000f, 0.000000f, 0.019129f),  //Pinky Tip joint
    };

    // skeleton entry is already done. just the nodes now.
    // setup a very simple node hierarchy for fingers
    // first setup the root node
    // 
    // root, This node has ID 0 and parent ID 0, to indicate it has no parent.
    SDKReturnCode t_Res = CoreSdk_AddNodeToSkeletonSetup(p_SklIndex, CreateNodeSetup(0, 0, 0, 0, 0, "Hand"));
    if (t_Res != SDKReturnCode::SDKReturnCode_Success)
    {
        return false;
    }

    // then loop for 5 fingers
    int t_FingerId = 0;
    for (uint32_t i = 0; i < t_NumFingers; i++)
    {
        uint32_t t_ParentID = 0;
        // then the digits of the finger that are linked to the root of the finger.
        for (uint32_t j = 0; j < t_NumJoints; j++)
        {
            t_Res = CoreSdk_AddNodeToSkeletonSetup(p_SklIndex, CreateNodeSetup(1 + t_FingerId + j, t_ParentID, t_Fingers[i * 4 + j].x, t_Fingers[i * 4 + j].y, t_Fingers[i * 4 + j].z, "fingerdigit"));
            if (t_Res != SDKReturnCode::SDKReturnCode_Success)
            {
                printf("Failed to Add Node To Skeleton Setup. The error given %d.", t_Res);
                return false;
            }
            t_ParentID = 1 + t_FingerId + j;
        }
        t_FingerId += t_NumJoints;
    }
    return true;
}

After adding the 20 nodes using CoreSdk_AddNodeToSkeletonSetup, a root node is added to define the start of the hand (wrist), which starts at the origin. Then, all the fingers and their joint nodes are added.

To set up the hand chains, we add 1 hand chain (ChainType_Hand) and 5 finger chains (ChainType_FingerIndex, ChainType_FingerMiddle, etc.) to define a hand.

For this example only a left hand is created. For the right hand, a similar process should be followed, where the finger nodes positions would be the other way around and the chain side would be configured to Side::Side_Right.

The hand chain contains settings to specify how it's root motion and rotation is controlled. In this example we're setting it to use IMU rotation by setting t_ChainSettings.hand.handMotion to HandMotion_IMU.

Once all this is done, the data is sent to MANUS Core via CoreSdk_LoadSkeleton(). This makes sure the skeleton is being animated and that the callback in the SDK Minimal Client will get updates.

Setup Hand Chains
/// @brief This function sets up some basic hand chains.
/// Chains are required for a Skeleton to be able to be animated, it basically tells Manus Core
/// which nodes belong to which body part and what data needs to be applied to which node.
/// @param p_SklIndex The index of the temporary skeleton on which the chains will be added.
/// @return Returns true if everything went fine, otherwise returns false.
bool SDKMinimalClient::SetupHandChains(uint32_t p_SklIndex)
{
    // Add the Hand chain, this identifies the wrist of the hand
    {
        ChainSettings t_ChainSettings;
        ChainSettings_Init(&t_ChainSettings);
        t_ChainSettings.usedSettings = ChainType::ChainType_Hand;
        t_ChainSettings.hand.handMotion = HandMotion::HandMotion_IMU;
        t_ChainSettings.hand.fingerChainIdsUsed = 5; //we will have 5 fingers
        t_ChainSettings.hand.fingerChainIds[0] = 1; //links to the other chains we will define further down
        t_ChainSettings.hand.fingerChainIds[1] = 2;
        t_ChainSettings.hand.fingerChainIds[2] = 3;
        t_ChainSettings.hand.fingerChainIds[3] = 4;
        t_ChainSettings.hand.fingerChainIds[4] = 5;

        ChainSetup t_Chain;
        ChainSetup_Init(&t_Chain);
        t_Chain.id = 0; //Every ID needs to be unique per chain in a skeleton.
        t_Chain.type = ChainType::ChainType_Hand;
        t_Chain.dataType = ChainType::ChainType_Hand;
        t_Chain.side = Side::Side_Left;
        t_Chain.dataIndex = 0;
        t_Chain.nodeIdCount = 1;
        t_Chain.nodeIds[0] = 0; //this links to the hand node created in the SetupHandNodes
        t_Chain.settings = t_ChainSettings;

        SDKReturnCode t_Res = CoreSdk_AddChainToSkeletonSetup(p_SklIndex, t_Chain);
        if (t_Res != SDKReturnCode::SDKReturnCode_Success)
        {
            return false;
        }
    }

    // Add the 5 finger chains
    const ChainType t_FingerTypes[5] = { ChainType::ChainType_FingerThumb,
        ChainType::ChainType_FingerIndex,
        ChainType::ChainType_FingerMiddle,
        ChainType::ChainType_FingerRing,
        ChainType::ChainType_FingerPinky };
    for (int i = 0; i < 5; i++)
    {
        ChainSettings t_ChainSettings;
        ChainSettings_Init(&t_ChainSettings);
        t_ChainSettings.usedSettings = t_FingerTypes[i];
        t_ChainSettings.finger.handChainId = 0; //This links to the wrist chain above.
        //This identifies the metacarpal bone, if none exists, or the chain is a thumb it should be set to -1.
        //The metacarpal bone should not be part of the finger chain, unless you are defining a thumb which does need it.
        t_ChainSettings.finger.metacarpalBoneId = -1;
        t_ChainSettings.finger.useLeafAtEnd = false; //this is set to true if there is a leaf bone to the tip of the finger.
        ChainSetup t_Chain;
        ChainSetup_Init(&t_Chain);
        t_Chain.id = i + 1; //Every ID needs to be unique per chain in a skeleton.
        t_Chain.type = t_FingerTypes[i];
        t_Chain.dataType = t_FingerTypes[i];
        t_Chain.side = Side::Side_Left;
        t_Chain.dataIndex = 0;
        if (i == 0) // Thumb
        {
            t_Chain.nodeIdCount = 4; //The amount of node id's used in the array
            t_Chain.nodeIds[0] = 1; //this links to the hand node created in the SetupHandNodes
            t_Chain.nodeIds[1] = 2; //this links to the hand node created in the SetupHandNodes
            t_Chain.nodeIds[2] = 3; //this links to the hand node created in the SetupHandNodes
            t_Chain.nodeIds[3] = 4; //this links to the hand node created in the SetupHandNodes
        }
        else // All other fingers
        {
            t_Chain.nodeIdCount = 4; //The amount of node id's used in the array
            t_Chain.nodeIds[0] = (i * 4) + 1; //this links to the hand node created in the SetupHandNodes
            t_Chain.nodeIds[1] = (i * 4) + 2; //this links to the hand node created in the SetupHandNodes
            t_Chain.nodeIds[2] = (i * 4) + 3; //this links to the hand node created in the SetupHandNodes
            t_Chain.nodeIds[3] = (i * 4) + 4; //this links to the hand node created in the SetupHandNodes
        }
        t_Chain.settings = t_ChainSettings;

        SDKReturnCode t_Res = CoreSdk_AddChainToSkeletonSetup(p_SklIndex, t_Chain);
        if (t_Res != SDKReturnCode::SDKReturnCode_Success)
        {
            return false;
        }
    }
    return true;
}

Skeleton updates

In the Run loop the client waits for a new update of skeleton data via the callback mechanism. Due to threading this is secured by a mutex and it is highly advised to copy the data only inside the mutex and do any other animation work outside the mutex scope.

In this example we don’t do anything with that data except post a console output message that new data is received. And sleep so other (network)processes get a chance to perform their duties.

Warning

The sleep time is set to 33 milliseconds, which is roughly 30 frames per second. The gloves can update at rates upwards of a 120 frames per second, so when you're looking to capture all the data, you should adjust the sleep time accordingly.

Callback Mechanism
/// @brief This gets called when the client is connected to manus core
/// @param p_SkeletonStreamInfo contains the meta data on how much data regarding the skeleton we need to get from the SDK.
void SDKMinimalClient::OnSkeletonStreamCallback(const SkeletonStreamInfo* const p_SkeletonStreamInfo)
{
    if (s_Instance)
    {
        ClientSkeletonCollection* t_NxtClientSkeleton = new ClientSkeletonCollection();
        t_NxtClientSkeleton->skeletons.resize(p_SkeletonStreamInfo->skeletonsCount);

        for (uint32_t i = 0; i < p_SkeletonStreamInfo->skeletonsCount; i++)
        {
            CoreSdk_GetSkeletonInfo(i, &t_NxtClientSkeleton->skeletons[i].info);
            t_NxtClientSkeleton->skeletons[i].nodes = new SkeletonNode[t_NxtClientSkeleton->skeletons[i].info.nodesCount];
            CoreSdk_GetSkeletonData(i, t_NxtClientSkeleton->skeletons[i].nodes, t_NxtClientSkeleton->skeletons[i].info.nodesCount);
        }
        s_Instance->m_SkeletonMutex.lock();
        if (s_Instance->m_NextSkeleton != nullptr) delete s_Instance->m_NextSkeleton;
        s_Instance->m_NextSkeleton = t_NxtClientSkeleton;
        s_Instance->m_SkeletonMutex.unlock();
    }
}

Since skeletal animation data is transmitted via a network, this will be received asynchronously in a different thread. It is not advised to hold up that thread longer then is necessary to copy the data.

For this a mutex is used so the data is thread safe and then copied.