Skip to content

License requirement

The functionality described requires a MANUS Bodypack or a MANUS license dongle with one of the following licenses: Core Pro Core XR Core Xsens Pro Core Qualisys Pro Core OptiTrack Pro Demo, or a Feature license with the SDK feature enabled.

SDK Minimal Client example

Introduction

The SDK Minimal client is an example client that demonstrates the basic functionality of the MANUS SDK. It will go over what is minimally required to get the SDK up and running and demonstrate how to receive raw skeleton data from all connected gloves. The SDKMinimalClient covers the following steps:

  1. Initialization:

    • The SDK is initialized using the CoreSdk_Initialize(SessionType::SessionType_CoreSDK, t_Remote) function call. This sets up the session type and specifies whether the client will work in integrated or remote mode.
    • The coordinate system for the client is set using the CoreSdk_InitializeCoordinateSystemWithVUH() function.

  2. Connection:

    • When not running in integrated mode. The client attempts to connect locally by calling the ConnectLocally() function in a loop until successful.
    • Once connected, the client sets the RawSkeletonHandMotion to auto.
    • In the main loop, it checks for new raw skeleton data and prints the first node position and rotation.

  3. Raw Skeleton Stream Callback:

    • The PrintRawSkeletonNodeInfo function demonstrates how to interpret the data coming in from the RawSkeletonStream.
    • It prints the position and rotation of the first node in the first skeleton and interprets and prints the raw skeleton data.

The SDK Minimal client is composed of the ManusSDK library. For Windows this is the ManusSDK.dll and for Linux libManusSDK.so or libManusSDK_Integrated.so. The SDKMinimalClient.cpp files and some headers found in the include folder.

libManusSDK_Integrated.so

The libManusSDK_Integrated.so can only be used for the integrated mode of the SDK, and is not able to connect to a MANUS Core on the network. The libManusSDK.so can be used for both the remote and integrated modes of the SDK. It's advantageous to use the libManusSDK_Integrated.so to get around the specific dependency requirements of the libManusSDK.so.

Initialize

Before using any functionality of the SDK, it is necessary to initialize it. This ensures that the system is set up correctly and ready for use.

The PlatformSpecificInitialization() function is an initializer used that prepares console output for the specific OS, but it is not required in a non-console environment.

The app then prompts the user what mode they would like to run the app in.

  • Core Integrated mode: The SDK is integrated into the client application.
  • Core Local mode: The SDK will connect to a MANUS Core running locally on this machine.
  • Core Remote mode: The SDK will search and connected to a MANUS Core instance on the network.

During the InitializeSDK() function, the SDK is initialized and sets the mode to integrated or remote and specifies the type of session. Typically, this will be of type SessionType_CoreSDK.

Next, the necessary callbacks are registered using the RegisterAllCallbacks() function. For this example only the CoreSdk_RegisterCallbackForRawSkeletonStream is registered. It is important to note that there are additional callbacks beyond this one, but they are not included in this example. For more information on other callbacks, please refer to the SDK Client article.

After registering the callbacks, the coordinate system used by the client is set using the CoreSdk_InitializeCoordinateSystemWithVUH() function. In this example, a z-up, x-positive, right-handed coordinate system is used. It is recommended to align the coordinate system with the application being developed, as this will make it easier to work with the data by letting MANUS Core handle the conversion.

Initialize
There are two ways to set up the coordinate system: using the VUH (View, Up, Handedness) system or using the Direction system.

``` C++ title="Coordinate system initialization"    
CoordinateSystemVUH t_VUH;
CoordinateSystemVUH_Init(&t_VUH);
t_VUH.handedness = Side::Side_Right;
t_VUH.up = AxisPolarity::AxisPolarity_PositiveZ;
t_VUH.view = AxisView::AxisView_XFromViewer;
t_VUH.unitScale = 1.0f; //1.0 is meters, 0.01 is cm, 0.001 is mm.

// The above specified coordinate system is used to initialize and the coordinate space is specified (world vs local).
const SDKReturnCode t_CoordinateResult = CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true);

// this is an example of an alternative way of setting up the coordinate system instead of VUH (view, up, handedness)
CoordinateSystemDirection t_Direction;
t_Direction.x = AxisDirection::AD_Right;
t_Direction.y = AxisDirection::AD_Up;
t_Direction.z = AxisDirection::AD_Forward;
const SDKReturnCode t_InitializeResult = CoreSdk_InitializeCoordinateSystemWithDirection(t_Direction, true);

The unit scale is specified as a float value. A scale of 1 represents meters, 0.01 represents centimeters, and 0.001 represents millimeters.

The second parameter of the CoreSdk_InitializeCoordinateSystemWithVUH() function indicates whether the coordinates should be set as world coordinates or relative coordinates. In this case, it is set to false to use relative coordinates.

Once the initialization is complete, the SDK is ready to be used, but it is not yet connected to an instance of MANUS Core.

Connection

When running the example it follows the following command structure:

  1. Connect when not running integrated, if not successful, just wait and retry.
  2. Set the hand motion mode to auto. Auto will make the hand move based on available tracking data. If no trackers are available IMU rotation will be used. This can alternatively be set to any of the HandMotion enum values (HandMotion_None, HandMotion_Auto, HandMotion_Tracker, HandMotion_Tracker_RotationOnly, HandMotion_IMU).
  3. Start the main loop.
  4. Check if there is new raw skeleton data.
  5. If there is, signal that there is and print the first node's position and rotation.
  6. In case of the first time data is received, some information about the node structure is printed.
  7. Wait for the escape key to exit.
Run
void SDKMinimalClient::Run()
{
    // first loop until we get a connection
    m_ConnectionType == ConnectionType::ConnectionType_Integrated ?
        ClientLog::print("minimal client is running in integrated mode.") :
        ClientLog::print("minimal client is connecting to MANUS Core. (make sure it is running)");

    while (Connect() != ClientReturnCode::ClientReturnCode_Success)
    {
        // not yet connected. wait
        ClientLog::print("minimal client could not connect.trying again in a second.");
        std::this_thread::sleep_for(std::chrono::milliseconds(1000));
    }

    if (m_ConnectionType != ConnectionType::ConnectionType_Integrated)
        ClientLog::print("minimal client is connected, setting up skeletons.");

    // set the hand motion mode of the RawSkeletonStream. This is optional and can be set to any of the HandMotion enum values. Default = None
    // auto will make it move based on available tracking data. If none is available IMU rotation will be used.
    const SDKReturnCode t_HandMotionResult = CoreSdk_SetRawSkeletonHandMotion(HandMotion_Auto);
    if (t_HandMotionResult != SDKReturnCode::SDKReturnCode_Success)
    {
        ClientLog::error("Failed to set hand motion mode. The value returned was {}.", (int32_t)t_HandMotionResult);
    }

    while (m_Running)
    {
        // check if there is new data available.
        m_RawSkeletonMutex.lock();

        delete m_RawSkeleton;
        m_RawSkeleton = m_NextRawSkeleton;
        m_NextRawSkeleton = nullptr;

        m_RawSkeletonMutex.unlock();

        if (m_RawSkeleton != nullptr && m_RawSkeleton->skeletons.size() != 0)
        {
            // print whenever new data is available
            ClientLog::print("raw skeleton data obtained for frame: {}.", std::to_string(m_FrameCounter));
            PrintRawSkeletonNodeInfo();
            m_FrameCounter++;
        }

        std::this_thread::sleep_for(std::chrono::milliseconds(33)); // Roughly 30fps, good enough to show the results, but too slow to retrieve all data.

        if (GetKeyDown(' ')) // press space to exit
        {
            m_Running = false;
        }
    }
}

The Connect function in the SDK Minimal Client attempts to establish a connection to a MANUS Core instance. This is only relevant when not running in integrated mode. The function performs the following steps:

  1. Look for Hosts:

    • The function starts by calling CoreSdk_LookForHosts to search for available MANUS Core hosts. It searches locally if the connection type is set to ConnectionType_Local.
    • If the search fails, the function returns ClientReturnCode_FailedToFindHosts.

  2. Retrieve Number of Hosts:

    • The function retrieves the number of available hosts found using CoreSdk_GetNumberOfAvailableHostsFound.
    • If the retrieval fails or no hosts are found, the function returns ClientReturnCode_FailedToFindHosts.

  3. Get Available Hosts:

    • The function allocates memory for the available hosts and retrieves their information using CoreSdk_GetAvailableHostsFound.
    • If the retrieval fails, the function returns ClientReturnCode_FailedToFindHosts.

  4. Host Selection:

    • If not connecting locally and multiple hosts are found, the function prompts the user to select a host.
    • The user is asked to input the host number, and if the input is invalid, the function returns ClientReturnCode_FailedToConnect.

  5. Connect to Host:

    • The function attempts to connect to the selected host using CoreSdk_ConnectToHost.
    • If the connection fails, the function returns ClientReturnCode_FailedToConnect.

  6. Return Success:

    • If all steps are successful, the function returns ClientReturnCode_Success.
Connect
ClientReturnCode SDKMinimalClient::Connect()
{
    bool t_ConnectLocally = m_ConnectionType == ConnectionType::ConnectionType_Local;
    SDKReturnCode t_StartResult = CoreSdk_LookForHosts(1, t_ConnectLocally);
    if (t_StartResult != SDKReturnCode::SDKReturnCode_Success)
    {
        return ClientReturnCode::ClientReturnCode_FailedToFindHosts;
    }

    uint32_t t_NumberOfHostsFound = 0;
    SDKReturnCode t_NumberResult = CoreSdk_GetNumberOfAvailableHostsFound(&t_NumberOfHostsFound);
    if (t_NumberResult != SDKReturnCode::SDKReturnCode_Success)
    {
        return ClientReturnCode::ClientReturnCode_FailedToFindHosts;
    }

    if (t_NumberOfHostsFound == 0)
    {
        return ClientReturnCode::ClientReturnCode_FailedToFindHosts;
    }

    std::unique_ptr<ManusHost[]> t_AvailableHosts; 
    t_AvailableHosts.reset(new ManusHost[t_NumberOfHostsFound]);

    SDKReturnCode t_HostsResult = CoreSdk_GetAvailableHostsFound(t_AvailableHosts.get(), t_NumberOfHostsFound);
    if (t_HostsResult != SDKReturnCode::SDKReturnCode_Success)
    {
        return ClientReturnCode::ClientReturnCode_FailedToFindHosts;
    }

    uint32_t t_HostSelection = 0;
    if (!t_ConnectLocally && t_NumberOfHostsFound > 1)
    {
        ClientLog::print("Select which host you want to connect to (and press enter to submit)");
        for (size_t i = 0; i < t_NumberOfHostsFound; i++)
        {
            auto t_HostInfo = t_AvailableHosts[i];
            ClientLog::print("[{}] hostname: , IP address: {}, version {}.{}.{}", i + 1, t_HostInfo.hostName, t_HostInfo.ipAddress, t_HostInfo.manusCoreVersion.major, t_HostInfo.manusCoreVersion.minor, t_HostInfo.manusCoreVersion.patch);
        }
        uint32_t t_HostSelectionInput = 0;
        std::cin >> t_HostSelectionInput;
        if (t_HostSelectionInput <= 0 || t_HostSelectionInput > t_NumberOfHostsFound)
            return ClientReturnCode::ClientReturnCode_FailedToConnect;

        t_HostSelection = t_HostSelectionInput - 1;
    }

    SDKReturnCode t_ConnectResult = CoreSdk_ConnectToHost(t_AvailableHosts[t_HostSelection]);

    if (t_ConnectResult == SDKReturnCode::SDKReturnCode_NotConnected)
    {
        return ClientReturnCode::ClientReturnCode_FailedToConnect;
    }

    return ClientReturnCode::ClientReturnCode_Success;  
}

Raw Skeleton Stream Callback

The PrintRawSkeletonNodeInfo function demonstrates how to interpret the data coming in from the RawSkeletonStream. It will print the position and rotation of the first node in the first skeleton, as well as interpreting and printing the raw skeleton data. Here's a detailed explanation of the function:

  1. Initial Check:

    • The function first checks if the m_RawSkeleton is nullptr or if the skeletons vector is empty. If either condition is true, it returns immediately.
    • If the skeletons vector is not empty and the first skeleton has nodes, it prints the position and rotation of the first node in the first skeleton.

  2. Interpreting Raw Skeleton Data:

    • The function retrieves the gloveId and the node count for the first skeleton.
    • It then calls CoreSdk_GetRawSkeletonNodeCount to get the number of nodes in the skeleton. If the call fails, an error message is printed, and the function returns.

  3. Getting Hierarchy Data:

    • The function allocates memory for an array of NodeInfo structures to hold the hierarchy data.
    • It calls CoreSdk_GetRawSkeletonNodeInfoArray to fill the array with the hierarchy data. If the call fails, an error message is printed and the function returns.

  4. Printing Node Information:

    • The function prints the glove data and the node information for each node in the skeleton. The information includes the node ID, side, chain type, finger joint type, and parent node ID. More information on the NodeInfo structure can be found in the Skeleton article.
    • After printing the information, the allocated memory for the NodeInfo array is deleted, and the m_PrintedNodeInfo flag is set to true as to only do this once.
PrintRawSkeletonNodeInfo
void SDKMinimalClient::PrintRawSkeletonNodeInfo()
{
    if ((m_RawSkeleton == nullptr || m_RawSkeleton->skeletons.size() == 0) || m_PrintedNodeInfo)
    {
        if (m_RawSkeleton->skeletons.size() != 0 && m_RawSkeleton->skeletons[0].nodes.size() != 0) {

            // prints the position and rotation of the first node in the first skeleton
            ManusVec3 t_Pos = m_RawSkeleton->skeletons[0].nodes[0].transform.position;
            ManusQuaternion t_Rot = m_RawSkeleton->skeletons[0].nodes[0].transform.rotation;

            ClientLog::print("Node 0 Position: x {} y {} z {} Rotation: x {} y {} z {} w {}", t_Pos.x, t_Pos.y, t_Pos.z, t_Rot.x, t_Rot.y, t_Rot.z, t_Rot.w);
        }
        return;
    }

    // this section demonstrates how to interpret the raw skeleton data.
    // how to get the hierarchy of the skeleton, and how to know bone each node represents.

    uint32_t t_GloveId = 0;
    uint32_t t_NodeCount = 0;

    t_GloveId = m_RawSkeleton->skeletons[0].info.gloveId;
    t_NodeCount = 0;

    SDKReturnCode t_Result = CoreSdk_GetRawSkeletonNodeCount(t_GloveId, t_NodeCount);
    if (t_Result != SDKReturnCode::SDKReturnCode_Success)
    {
        ClientLog::error("Failed to get Raw Skeleton Node Count. The error given was {}.", (int32_t)t_Result);
        return;
    }

    // now get the hierarchy data, this needs to be used to reconstruct the positions of each node in case the user set up the system with a local coordinate system.
    // having a node position defined as local means that this will be related to its parent. 

    NodeInfo* t_NodeInfo = new NodeInfo[t_NodeCount];
    t_Result = CoreSdk_GetRawSkeletonNodeInfoArray(t_GloveId, t_NodeInfo, t_NodeCount);
    if (t_Result != SDKReturnCode::SDKReturnCode_Success)
    {
        ClientLog::error("Failed to get Raw Skeleton Hierarchy. The error given was {}.", (int32_t)t_Result);
        return;
    }

    ClientLog::print("Received Skeleton glove data from Core. skeletons:{} first skeleton glove id:{}", m_RawSkeleton->skeletons.size(), m_RawSkeleton->skeletons[0].info.gloveId);
    ClientLog::print("Printing Node Info:");


    // prints the information for each node, the chain type will which part of the body it is. The finger joint type will be which bone of the finger it is.
    for (size_t i = 0; i < t_NodeCount; i++)
    {
        ClientLog::printWithPadding("Node ID: {} Side: {} ChainType: {} FingerJointType: {}, Parent Node ID: {}",2, std::to_string(t_NodeInfo[i].nodeId), t_NodeInfo[i].side, t_NodeInfo[i].chainType, t_NodeInfo[i].fingerJointType, std::to_string(t_NodeInfo[i].parentId));
    }

    delete[] t_NodeInfo;
    m_PrintedNodeInfo = true;
}