License requirement
The functionality described requires a MANUS Bodypack or a MANUS license key with the SDK feature enabled.
SDK Client example
Introduction
The SDK Client example functions as a demonstration of all major SDK functions. It is an interactive terminal application that shows how to connect to MANUS Core, how to set up the coordinate system and how to handle the landscape, gloves, dongles, skeletons, gestures, trackers and time tracking. The example client is not meant to be a full-fledged application, but rather a demonstration of how to use the SDK functions. The example client is written in C++ and uses the MANUS Core SDK.
All SDK usage in the example lives in the sdk/ folder: plain C++ modules that contain no UI code and are written to be copied into your own project. The ui/ folder (the terminal screens) and the app/ folder (the application shell) only call into these modules, never into the SDK directly — a build step (cmake/CheckLayers.cmake) enforces this, so the sdk/ modules stay self-contained and copyable. Most code snippets in this article therefore come from the sdk/ modules; a few come from ui/screens/ where the interesting part is how a screen drives those modules. The file each snippet comes from is noted in its title.
Building the SDK Client
The SDK Client is built with CMake on both Windows and Linux. To build the client you need:
CMake3.16 or newer- A C++17 compiler (Visual Studio/MSVC on Windows, GCC or Clang on Linux)
From inside the C++/SDKClient folder of the SDK package, run:
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release
These two commands are the same on both platforms. -DCMAKE_BUILD_TYPE=Release is needed on Linux, where the default generator (Unix Makefiles) picks the build type at configure time; --config Release is what the multi-config Visual Studio generator on Windows uses instead.
The MANUS SDK headers and library are picked up from the SDK package automatically. If you moved them, point the build at them with -DMANUS_SDK_DIR=....
On Linux, a number of packages are required and the build automatically selects the SDK library matching your processor architecture — lib/<arch>/libManusSDK-<arch>.so for amd64 or aarch64. See the Linux guide for details.
The resulting executable also accepts two non-interactive flags, which are handy when checking a fresh toolchain or terminal:
SDKClient --smoke: renders one frame, initializes and shuts the SDK down again, and printsSMOKE PASS/SMOKE FAIL. No MANUS Core required.SDKClient --probe: drives the Skeletons screen with synthetic key events, headless, and prints a pass/fail line per check.
Without a flag, SDKClient starts the interactive client. Any ANSI/VT-capable terminal will do (Windows Terminal, conhost on Windows 10+, the common Linux terminals).
SDK Callbacks
All the data streams MANUS Core provides are structured as callbacks. A callback is a function that you can pass into a CoreSdk_RegisterCallback... function. These functions call them at another point (on another thread) when a certain event occurs. This way users do not need to poll the SDK to figure out if there is new data available.
It is good practice to register all the callbacks you require after initializing the SDK and before connecting to MANUS Core. You only need to register the callbacks which you intend to use, if you do not plan to use certain data streams this could decrease the traffic from MANUS Core. In the example client every sdk/ module exposes a small Register function for its stream, and they are all registered right after initializing the SDK:
manus::SdkResult t_Result = manus::Connection::Initialize(p_Mode);
if (t_Result.Ok()) t_Result = manus::Logs::Register();
if (t_Result.Ok()) t_Result = manus::LandscapeCache::Register();
if (t_Result.Ok()) t_Result = manus::GestureStream::Register();
if (t_Result.Ok()) t_Result = manus::StreamTime::Register();
if (t_Result.Ok()) t_Result = manus::Trackers::RegisterStream();
if (t_Result.Ok()) t_Result = manus::SkeletonsRetargeted::RegisterStream();
if (t_Result.Ok()) t_Result = manus::SkeletonsRaw::RegisterStream();
if (t_Result.Ok()) t_Result = manus::RawDeviceStream::Register();
Two of these deserve a note. Logs::Register registers the SDK's own log callback (CoreSdk_RegisterCallbackForOnLog); the client keeps those lines in a buffer and shows them in the SDK log pane that sits at the bottom of every screen, which is the first place to look when something does not connect. StreamTime::Register registers the ergonomics-stream callback, which feeds both the ergonomics buffer and the publish timestamp used by the Timecode screen — so there is no separate Ergonomics::Register.
As an example of a callback, this is how the ergonomics stream is received:
void OnErgonomicsStreamCallback(const ErgonomicsStream* const p_Stream)
{
FeedTimestamp(p_Stream->publishTime);
for (uint32_t i = 0; i < p_Stream->dataCount; ++i)
{
if (p_Stream->data[i].isUserID) continue;
std::lock_guard<std::mutex> t_Lock(g_ErgonomicsMutex);
g_ErgonomicsData[p_Stream->data[i].id] = p_Stream->data[i];
}
}
This callback is called whenever the SDK receives new ergonomic data from MANUS Core. A best practice for data being received on another thread (which happens with the SDK's callback mechanism) is to save the data somewhere and then process whatever you want to do with the data on your own thread. This is exactly what the example does: the callback only stores the latest data per glove in a mutex-protected buffer, and the UI thread later fetches a copy via Ergonomics::LatestForGlove. This way you do not block/delay the callback thread, which could lead to delayed data transfer.
In several of our streams, we add timestamps to show when the data was captured. To get a more readable timestamp you need to call the CoreSdk_GetTimestampInfo on the timestamp value, this will translate the timestamp value to a date & time format.
Client Connection
There are two modes in which the SDK can be initialized. Integrated mode and Remote mode. In Integrated mode, the SDK will talk directly to the gloves without the need for a MANUS Core instance. In Remote mode, the SDK connects to a MANUS Core instance which will then talk to the gloves. The Integrated mode is usually used for applications that want to handle all glove functionality themselves and deeply integrate the MANUS gloves into their application. The Remote mode is usually used for applications that want to use the MANUS Core Dashboard to handle the gloves and dongles and only want to receive the data from the gloves.
On startup the client asks "How do you want to connect?" and offers three options
- `Core Integrated` mode - run standalone, no seperate MANUS Core needed
- `Core Local` mode - connect to the MANUS Core running on this machine
- `Core Remote` mode - search the network for MANUS Core hosts
Core Local is preselected. If a mode fails to come up the client shows the reason (for example Failed to connect: ... or No MANUS Core hosts found.) and offers [R] to retry.
The SDK is initialized differently depending on the chosen mode: CoreSdk_InitializeIntegrated runs MANUS Core inside the client process, while CoreSdk_InitializeCore prepares the SDK to talk to a separately running MANUS Core instance. The Connection::Initialize function shows this, together with registering the connection callbacks and setting up the coordinate system (explained in the next section):
SdkResult Connection::Initialize(ConnectionMode p_Mode)
{
{
std::lock_guard<std::mutex> t_Lock(g_Mutex);
g_Mode = p_Mode;
}
// Integrated runs Core inside this process; the other modes talk to a
// separately running MANUS Core.
SDKReturnCode t_Result = p_Mode == ConnectionMode::Integrated
? CoreSdk_InitializeIntegrated()
: CoreSdk_InitializeCore();
if (t_Result != SDKReturnCode_Success) return t_Result;
g_Initialized = true;
t_Result = CoreSdk_RegisterCallbackForOnConnect(*OnConnectedCallback);
if (t_Result != SDKReturnCode_Success) return t_Result;
t_Result = CoreSdk_RegisterCallbackForOnDisconnect(*OnDisconnectedCallback);
if (t_Result != SDKReturnCode_Success) return t_Result;
// Without a coordinate system the SDK refuses to connect to any host.
// This client uses: z-up, x-from-viewer, right-handed, meter scale.
// (Unity, Unreal etc. each use their own; set what matches your app.)
CoordinateSystemVUH t_VUH;
CoordinateSystemVUH_Init(&t_VUH);
t_VUH.handedness = Side_Right;
t_VUH.up = AxisPolarity_PositiveZ;
t_VUH.view = AxisView_XFromViewer;
t_VUH.unitScale = 1.0f;
return CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true);
}
When using Remote, the network is scanned, and a list of available MANUS Core instances is returned. To explore the code behind our example client, consider the following snippet for finding hosts:
SdkResult Connection::ScanForHosts(std::vector<ManusHost>& p_OutHosts, uint32_t p_WaitSeconds)
{
p_OutHosts.clear();
bool t_LoopbackOnly;
{
std::lock_guard<std::mutex> t_Lock(g_Mutex);
t_LoopbackOnly = g_Mode == ConnectionMode::Local;
}
// Blocks while hosts get a chance to reply to the search.
SDKReturnCode t_Result = CoreSdk_LookForHosts(p_WaitSeconds, t_LoopbackOnly);
if (t_Result != SDKReturnCode_Success) return t_Result;
uint32_t t_HostCount = 0;
t_Result = CoreSdk_GetNumberOfAvailableHostsFound(&t_HostCount);
if (t_Result != SDKReturnCode_Success) return t_Result;
if (t_HostCount == 0) return SDKReturnCode_Success;
p_OutHosts.resize(t_HostCount);
t_Result = CoreSdk_GetAvailableHostsFound(p_OutHosts.data(), t_HostCount);
if (t_Result != SDKReturnCode_Success) p_OutHosts.clear();
return t_Result;
}
To find all the available MANUS Core instances, the CoreSdk_LookForHosts function needs to be called. This function call is a blocking function and therefore will block the calling thread for the specified duration. We advise not running any time sensitive functions or UI in the same thread as the Manus SDK connection.
After this function has completed you can get the number of hosts found by calling the CoreSdk_GetNumberOfAvailableHostsFound function. If this function returns 0 hosts, either no MANUS core is available, or the network is being blocked by a security system or another program.
The next step is creating an array at least the size of the number of found hosts and passing this array into the CoreSdk_GetAvailableHostsFound function. This function will put information about all the hosts that are found on the network into the array.
Once you have at least one host to connect to you can connect to it with code like this:
SdkResult Connection::Connect(const ManusHost& p_Host)
{
return CoreSdk_ConnectToHost(p_Host);
}
SdkResult Connection::ConnectIntegrated()
{
// The in-process Core is addressed with an empty host struct.
ManusHost t_Empty;
ManusHost_Init(&t_Empty);
return CoreSdk_ConnectToHost(t_Empty);
}
The chosen host structure needs to be put into the CoreSdk_ConnectToHost function. This function then attempts to connect to the given host and will trigger the OnConnectedCallback callback when it is successful. This callback is registered via the CoreSdk_RegisterCallbackForOnConnect function.
If a connection to MANUS Core is lost the OnDisconnected callback is invoked. This one can be registered via the CoreSdk_RegisterCallbackForOnDisconnect function. CoreSdk_GetIsConnectedToCore can be polled for the same state where a callback is inconvenient, which is what the example client's header uses to show Connected/Not connected.
When shutting down, call CoreSdk_ShutDown — the example client does this once, on exit, via Connection::Shutdown. Note that the mode is fixed at initialization time, so the client asks for it before initializing and does not offer a way to change it afterwards.
In the example client the connected callback also checks whether the SDK and MANUS Core versions are compatible and stores the session ID:
void OnConnectedCallback(const ManusHost* const p_Host)
{
// Optional, but worth showing: an SDK/Core version mismatch is a
// common cause of confusing behavior.
ManusVersion t_SdkVersion{}, t_CoreVersion{};
bool t_IsCompatible = false;
const bool t_HaveVersions = CoreSdk_GetVersionsAndCheckCompatibility(
&t_SdkVersion, &t_CoreVersion, &t_IsCompatible) == SDKReturnCode_Success;
uint32_t t_SessionId = 0;
CoreSdk_GetSessionId(&t_SessionId);
{
std::lock_guard<std::mutex> t_Lock(g_Mutex);
g_Info.connected = true;
g_Info.hostName.assign(p_Host->hostName,
strnlen(p_Host->hostName, MAX_NUM_CHARS_IN_HOST_NAME));
g_Info.sdkVersion = t_HaveVersions ? t_SdkVersion.versionInfo : "";
g_Info.coreVersion = t_HaveVersions ? t_CoreVersion.versionInfo : "";
g_Info.versionsCompatible = t_IsCompatible;
g_Info.sessionId = t_SessionId;
}
Logs::Push("Connected to MANUS Core.");
if (t_HaveVersions && !t_IsCompatible)
Logs::Push("WARNING: SDK and Core versions are not compatible.");
NotifyChanged();
}
Coordinate system Initialization
After registering the callbacks, the coordinate system used by the client is set using the CoreSdk_InitializeCoordinateSystemWithVUH() function, as shown at the end of Connection::Initialize. In this example, a z-up, x-from-viewer, right-handed coordinate system with meter scale is used. It is recommended to align the coordinate system with the application being developed, as this will make it easier to work with the data by letting MANUS Core handle the conversion.
There are two ways to set up the coordinate system: using the VUH (View, Up, Handedness) system or using the Direction system (CoreSdk_InitializeCoordinateSystemWithDirection). The example client uses VUH:
// Without a coordinate system the SDK refuses to connect to any host.
// This client uses: z-up, x-from-viewer, right-handed, meter scale.
// (Unity, Unreal etc. each use their own; set what matches your app.)
CoordinateSystemVUH t_VUH;
CoordinateSystemVUH_Init(&t_VUH);
t_VUH.handedness = Side_Right;
t_VUH.up = AxisPolarity_PositiveZ;
t_VUH.view = AxisView_XFromViewer;
t_VUH.unitScale = 1.0f;
return CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true);
The unit scale is specified as a float value. A scale of 1 represents meters, 0.01 represents centimeters, and 0.001 represents millimeters.
The second parameter of the CoreSdk_InitializeCoordinateSystemWithVUH() function indicates whether the coordinates should be set as world coordinates or relative coordinates. In this example, it is set to true to use world coordinates.
Main menu
Once connected, the client shows a menu of screens, each covering one area of the SDK:
| Screen | What it covers |
|---|---|
| Devices | devices: pairing, haptics, firmware, the dongle's license type |
| Users | the user system: add, rename, remove, order, assign gloves and dongles |
| Calibration | the glove calibration wizard and saving/loading .mcal files |
| Skeletons | raw, retargeted and temporary skeletons |
| Trackers | live tracker poses, assignment, offsets, timeouts, tracking-system settings |
| Ergonomics | the per-glove joint angles from the ergonomics stream |
| Gestures | the gesture stream, with ids resolved to names from the landscape |
| Timecode | the timecode state and the decoded stream timestamp |
| License | writing a .lic file to a dongle and retrieving the newest license |
| Landscape Browser | the whole landscape structure, browsable |
Navigation is [Up/Down] and [Enter] to open a screen, [Q] to go back to the menu and [Esc] to quit. Every screen shows its own key hints on its bottom line. The SDK log pane is always visible underneath and scrolls with the mouse wheel wherever you are.
None of the code used to display or navigate through the menus is needed to use the SDK; opening a screen does not start or stop a stream, all data is received simultaneously from the moment the callbacks are registered.
Landscape
The landscape is a record of the current state of devices, users, trackers, gestures etcetera. The callback happens once every second. Our sample client does not use all the information given via the landscape to keep the example a bit easier to follow.
void OnLandscapeCallback(const Landscape* const p_Landscape)
{
// The gesture names live outside the landscape struct and must be
// fetched separately; the count comes from the landscape itself.
std::vector<GestureLandscapeData> t_Gestures(p_Landscape->gestureCount);
if (p_Landscape->gestureCount > 0)
CoreSdk_GetGestureLandscapeData(t_Gestures.data(), (uint32_t)t_Gestures.size());
std::function<void()> t_OnChanged;
{
std::lock_guard<std::mutex> t_Lock(g_Mutex);
g_Landscape = *p_Landscape;
g_Gestures = std::move(t_Gestures);
g_HasLandscape = true;
t_OnChanged = g_OnChanged;
}
if (t_OnChanged) t_OnChanged();
}
If you plan to use the gestures and need the landscape data related to these, make sure to call the CoreSdk_GetGestureLandscapeData to retrieve all the data needed for the gestures from within the callback to ensure that the entirety of the landscape is stored correctly.
The rest of the client never touches the landscape from the callback thread: it asks the cache for a copy via LandscapeCache::Snapshot() (or one of the convenience accessors such as GlovesSortedById()) whenever it needs the current state.
The example client includes a Landscape browser screen that lets you explore the entire landscape structure interactively, a convenient way to see what data is available while developing.
The gloves and dongle information can be found under the gloveDevices variable. You can find the battery information, the pairing state of the glove, the type of glove, haptics, etc. inside the GloveLandscapeData structure. In the DongleLandscapeData structure you can find information such as the dongle type, firmware versions, which gloves are paired to it, etc. In the users you can find information such as which dongle and gloves are assigned to which user.
Devices
The Devices screen of the example client lists the gloves and dongles from the landscape and shows the license type present on each dongle. From this screen gloves can be paired ([P], or [D] to pick a dongle) and unpaired ([U]), firmware updates can be started ([F], on gloves and dongles alike), and for haptics-enabled gloves a haptic test pattern can be played ([V]): each finger buzzes in turn, then all five ramp together, then everything is set back to zero. All of it goes through the glove ID to identify which glove needs to act (sdk/Devices.cpp).
The ergonomics callback (shown in the SDK Callbacks section) stores the latest data per glove ID; the dedicated Ergonomics screen displays it by asking for a copy:
bool Ergonomics::LatestForGlove(uint32_t p_GloveId, ErgonomicsData& p_OutData)
{
std::lock_guard<std::mutex> t_Lock(g_ErgonomicsMutex);
const auto t_Found = g_ErgonomicsData.find(p_GloveId);
if (t_Found == g_ErgonomicsData.end()) return false;
p_OutData = t_Found->second;
return true;
}
Haptics
MANUS Glove devices that support haptics can be controlled using two functions. CoreSdk_VibrateFingersForGlove vibrates the fingers of a specific GloveID; CoreSdk_VibrateFingersForSkeleton vibrates the fingers of a specific skeleton, taking a skeleton ID and a Side. Which method is easiest depends on the specific implementation — the example client uses the glove variant, since its Devices screen already works per glove. The powers array is an array of floats that represent the power of the vibration for each finger. The powers array should be of size 5, with each index representing a finger, starting at the thumb (0) and ending at the pinky (4) The powers should be between 0 and 1, where 0 is no vibration and 1 is the maximum vibration. After setting the vibration to a specific value, changing it requires a new call to the function. So to stop the vibration, you need to call the function with all the powers set to 0.
SdkResult Devices::VibrateFingers(uint32_t p_GloveId, const std::array<float, 5>& p_Powers)
{
return CoreSdk_VibrateFingersForGlove(p_GloveId, p_Powers.data());
}
Firmware updates
A firmware update for a dongle or glove can be requested with the CoreSdk_UpdateFirmware function, passing the ID of the device to update. The function returns Core's initial response to the request: whether the update was accepted, and a message describing the result. There is no progress stream, the further progress of the update is reported through the device's updateStatus in the landscape. The example wraps this in Firmware::Update (sdk/Firmware.cpp), which the Devices screen uses to start updates.
Note
Gloves can only be updated over a USB cable, updating a wirelessly connected glove is not supported.
Updating a license
A dongle's license can be written from a .lic file and retrieved online from the MANUS license service. In the example client both actions live in the License screen, backed by sdk/License.cpp.
Writing reads a .lic file and applies its contents to the dongle via CoreSdk_SetLicense:
SdkResult License::SetFromFile(uint32_t p_DongleId, const std::string& p_FilePath,
bool& p_OutAccepted, std::string& p_OutMessage)
{
p_OutAccepted = false;
p_OutMessage.clear();
std::ifstream t_File(p_FilePath, std::ios::binary);
if (!t_File)
{
p_OutMessage = "Could not read the file.";
return SDKReturnCode_InvalidArgument;
}
std::vector<char> t_Data(
(std::istreambuf_iterator<char>(t_File)), std::istreambuf_iterator<char>());
if (t_Data.empty())
{
p_OutMessage = "The file is empty.";
return SDKReturnCode_InvalidArgument;
}
Response t_Response{};
const SDKReturnCode t_Result = CoreSdk_SetLicense(
p_DongleId, t_Data.data(), (uint32_t)t_Data.size(), &t_Response);
p_OutAccepted = t_Response.result;
p_OutMessage = t_Response.message.message;
return t_Result;
}
Retrieving, via CoreSdk_RetrieveLicense, makes Core read the dongle's current license, present it to the license service, and write back the newest signed license:
SdkResult License::Retrieve(uint32_t p_DongleId, bool& p_OutAccepted, std::string& p_OutMessage)
{
p_OutAccepted = false;
p_OutMessage.clear();
Response t_Response{};
const SDKReturnCode t_Result = CoreSdk_RetrieveLicense(p_DongleId, &t_Response);
p_OutAccepted = t_Response.result;
p_OutMessage = t_Response.message.message;
return t_Result;
}
License service configuration
Retrieving a license requires Core to have a license service URL configured (the licenseServiceUrl setting). If the service is unreachable or rejects the request, the dongle keeps the license it already holds. On Linux this online exchange uses libcurl — see the Linux SDK guide for the required package.
Skeletons
MANUS Core uses skeletons to animate the hands and bodies. Skeletons consist of Nodes and Chains.
Nodes represent individual joints in the fingers and Chains represent groupings of Nodes. For example, your thumb chain would contain all the nodes that make up your thumb.
There are three different skeletons in MANUS Core, Temporary Skeletons, Retargeted Skeletons and Raw Skeletons. Temporary Skeletons are the skeletons that you create and send to the development tools for verification, but you do not load them into MANUS Core's retargeting system. Retargeted Skeletons are usually called Skeletons in the SDK, which are loaded into MANUS Core with the animation data applied. The Raw Skeletons are hand models on which MANUS Core applies the hand data without doing any retargeting, these are not necessarily using the same bone hierarchy structure as the skeletons that the user loads into MANUS Core.
Skeletal Data
MANUS Core supplies two streams of skeletal data. The first being retargeted skeletal data, this is the glove data applied onto a skeleton of the user's choosing. The second stream is the raw skeletal data, this is the glove data applied onto a standardized MANUS hand skeleton. Depending on your application's needs you can use one stream or the other, or in some cases even both.
The retargeted skeletal data is useful when you have your own skeleton or hand model, and you would like things like pinches to look better than they would when simply applying rotations from the raw skeleton stream. Using retargeted skeletal data allows you to create hands that have varying sizings and structures compared to the glove wearer's hand.
The raw skeletal data is generally the more accurate representation of the glove wearer's hand. The rotations and positions output in the raw skeleton stream are usually closer to the real world hand of a person, but applying this data directly to your desired skeleton or hand model can be a much more challenging task and will not guarantee lifelike animation. When wanting to include any wrist positioning and orientation in the raw skeleton stream the global source can be specified using the CoreSDK_SetRawSkeletonHandMotion() function. This can be set to any of the HandMotion enum values (HandMotion_None, HandMotion_Auto, HandMotion_Tracker, HandMotion_Tracker_RotationOnly, HandMotion_IMU).
HandMotion_None: No hand motion data is used.HandMotion_Auto: Automatically uses tracker data if a tracker is assigned to that hand in MANUS Core Dashboard; otherwise, it uses glove IMU data.HandMotion_Tracker: Uses a tracker's orientation and position if available.HandMotion_Tracker_RotationOnly: Uses the tracker's orientation without its position.HandMotion_IMU: Uses the gloves' IMU orientation.
Two further settings shape the raw stream, both read and written through the sdk/SkeletonsRaw.cpp module and both exposed on the raw view of the Skeletons screen:
- Pinch compensation (
CoreSdk_GetRawSkeletonPinchCompensation/CoreSdk_SetRawSkeletonPinchCompensation) is a boolean that corrects the finger tips while pinching, so tips meet where the real fingers meet. This applies to theMetaglove. - Casing compensation (
CoreSdk_GetRawSkeletonCasingCompensation/CoreSdk_SetRawSkeletonCasingCompensation) is a filter strength for theMetaglove Pro, passed as a float. The example client clamps it to the0.0–1.0range and steps it by0.1.
Both are global settings on the raw skeleton stream rather than per-glove, so a change takes effect for every glove the stream carries.
The closer the dimensions are to a real-life hand, the smaller the difference is between the retargeted and the raw skeletal data, at which point it tends to be much easier to just use the retargeted skeletal data for your skeleton or hand model. Retargeted skeletal data needs to be set up using the Skeleton system which requires more effort.
Both skeletal stream callbacks function in a similar way, and can be seen in the example client:
void OnSkeletonStreamCallback(const SkeletonStreamInfo* const p_Info)
{
// An update only contains skeletons whose data changed, so entries
// are merged into the buffer instead of replacing it wholesale.
for (uint32_t i = 0; i < p_Info->skeletonsCount; ++i)
{
SkeletonsRetargeted::LiveSkeleton t_Skeleton;
if (CoreSdk_GetSkeletonInfo(i, &t_Skeleton.info) != SDKReturnCode_Success) continue;
t_Skeleton.nodes.resize(t_Skeleton.info.nodesCount);
if (CoreSdk_GetSkeletonData(i, t_Skeleton.nodes.data(), t_Skeleton.info.nodesCount) != SDKReturnCode_Success)
continue;
const uint32_t t_Id = t_Skeleton.info.id;
std::lock_guard<std::mutex> t_Lock(g_Mutex);
g_Latest[t_Id] = std::move(t_Skeleton);
}
}
void OnRawSkeletonStreamCallback(const SkeletonStreamInfo* const p_Info)
{
for (uint32_t i = 0; i < p_Info->skeletonsCount; ++i)
{
SkeletonsRaw::RawSkeleton t_Skeleton;
if (CoreSdk_GetRawSkeletonInfo(i, &t_Skeleton.info) != SDKReturnCode_Success) continue;
t_Skeleton.nodes.resize(t_Skeleton.info.nodesCount);
if (CoreSdk_GetRawSkeletonData(i, t_Skeleton.nodes.data(), t_Skeleton.info.nodesCount) != SDKReturnCode_Success)
continue;
const uint32_t t_GloveId = t_Skeleton.info.gloveId;
std::lock_guard<std::mutex> t_Lock(g_Mutex);
g_RawSkeletons[t_GloveId] = std::move(t_Skeleton);
}
}
Getting the basic skeleton information such as the ID and node count can be done via the CoreSdk_GetSkeletonInfo or the CoreSdk_GetRawSkeletonInfo respectively. In this info you can get the ID of the skeleton, which is either the id of the retargeted skeleton or of the glove ID, depending on which of the two streams you are looking at.
Using the CoreSdk_GetSkeletonData or the CoreSdk_GetRawSkeletonData you can get all the node data for the skeleton. The ID of the node is either the ID you gave it when setting up the skeleton for the retargeting or the ID that we gave the raw skeleton in MANUS Core.
The difficult part about using the Raw Skeletons compared to the Retargeting Skeleton is that you will need to reconstruct the hierarchy of that skeleton to be able to apply the data gotten from the stream. The SkeletonsRaw::NodeHierarchy function demonstrates how to get the hierarchical information for the raw skeleton.
SdkResult SkeletonsRaw::NodeHierarchy(uint32_t p_GloveId, std::vector<NodeInfo>& p_OutNodes)
{
p_OutNodes.clear();
uint32_t t_Count = 0;
SDKReturnCode t_Result = CoreSdk_GetRawSkeletonNodeCount(p_GloveId, &t_Count);
if (t_Result != SDKReturnCode_Success) return t_Result;
p_OutNodes.resize(t_Count);
t_Result = CoreSdk_GetRawSkeletonNodeInfoArray(p_GloveId, p_OutNodes.data(), t_Count);
if (t_Result != SDKReturnCode_Success) p_OutNodes.clear();
return t_Result;
}
The CoreSdk_GetRawSkeletonNodeCount function can be used to get the number of nodes that exist in the raw skeleton of a certain glove. Using the CoreSdk_GetRawSkeletonNodeInfoArray you can get all the information for every node in the Raw Skeleton. This NodeInfo gives you information such as the parent's ID and the side. Using this information you can reconstruct the skeleton. When using a relative coordinate system (a not world space coordinate system) you will need to keep the hierarchy into account to be able to accurately reconstruct the skeleton.
The nodes point outwards in alignment with the "View" axis. Every node points at the subseqeuent node in the finger (MCP, PIP, DIP). When the fingers open and close, the nodes rotate around the "Side" axis. For more information on this, see the Coordinate System section:
Skeletons in the Sample Client
The Skeletons screen has three views, switched with [1], [2] and [3], one per kind of skeleton described above. Each view lists its own keys on the bottom line.
[1] Raw shows the per-glove raw skeleton as it streams in. [V] switches between a sketch of the hand and a table of node values, [H] cycles the wrist motion source through the HandMotion values, and [P] and [J]/[K] toggle pinch compensation and lower/raise casing compensation (see Skeletal Data).
[2] Retargeted is where skeletons are loaded and unloaded. [B] and [N] build and load the example left or right hand, [L] loads a skeleton from an .mskl file, and [M] unloads the selected skeleton from MANUS Core. [D] toggles "DevTools on load", which shares newly loaded skeletons with MANUS DevTools for 3D visualization. The example skeletons target user index 0, so the view also states whether that user exists and has gloves assigned — without them a loaded skeleton simply sits still.
[3] Temporary manages temporary skeletons: [B] builds the example setup and shares it with DevTools, [E] exports the selected one to an .mskl file, [F] imports one, [C] clears the selected one, and [X] clears the temporary skeletons of all sessions after a confirmation.
Setup a skeleton
To setup a hand skeleton and make sure it animates correctly, the following steps are required:
- Setup the basic skeleton information
- Create and add nodes for the skeleton
- Create and add chains for the skeleton
The first step is creating a SkeletonSetupInfo structure and setting the data you wish to adjust. For a hand skeleton the type needs to be set to the SkeletonType_Hand enumerator.
Please note that this example builds both Left or Right hand skeleton, depending on the side passed as a parameter (Side p_Side).
SdkResult SkeletonsRetargeted::CreateExampleHandSetup(Side p_Side, uint32_t& p_OutIndex)
{
p_OutIndex = 0;
SkeletonSetupInfo t_Setup;
SkeletonSetupInfo_Init(&t_Setup);
t_Setup.type = SkeletonType_Hand;
t_Setup.settings.scaleToTarget = true;
t_Setup.settings.useEndPointApproximations = true;
// Animate this skeleton with the gloves of the user at index 0.
// A skeleton targeting a non-existent user simply does not animate.
t_Setup.settings.targetType = SkeletonTargetType_UserIndexData;
t_Setup.settings.skeletonTargetUserIndexData.userIndex = 0;
std::snprintf(t_Setup.name, sizeof(t_Setup.name), "%s", p_Side == Side_Left ? "LeftHand" : "RightHand");
const SDKReturnCode t_Result = CoreSdk_CreateSkeletonSetup(t_Setup, &p_OutIndex);
if (t_Result != SDKReturnCode_Success) return t_Result;
if (!SetupHandNodes(p_OutIndex, p_Side)) return SDKReturnCode_Error;
if (!SetupHandChains(p_OutIndex, p_Side)) return SDKReturnCode_Error;
return SDKReturnCode_Success;
}
SdkResult SkeletonsRetargeted::LoadExampleHand(Side p_Side, bool p_SendToDevTools,
uint32_t p_SessionId, uint32_t& p_OutSkeletonId)
{
p_OutSkeletonId = 0;
uint32_t t_SetupIndex = 0;
SdkResult t_Result = CreateExampleHandSetup(p_Side, t_SetupIndex);
if (!t_Result.Ok()) return t_Result;
if (p_SendToDevTools)
{
// Saving the setup as a temporary skeleton makes it visible to the
// DevTools application for inspection/editing.
t_Result = CoreSdk_SaveTemporarySkeleton(t_SetupIndex, p_SessionId, true);
if (!t_Result.Ok()) return t_Result;
}
t_Result = CoreSdk_LoadSkeleton(t_SetupIndex, &p_OutSkeletonId);
if (!t_Result.Ok()) return t_Result;
if (p_OutSkeletonId == 0) return SDKReturnCode_Error;
std::lock_guard<std::mutex> t_Lock(g_Mutex);
g_LoadedIds.push_back(p_OutSkeletonId);
return SDKReturnCode_Success;
}
To have the skeleton move according to the hand movement of the first user you can set the settings.targetType to SkeletonTarget_UserIndexData and the settings.SkeletonTargetUserIndexData to 0. This means it will look at all the users in MANUS Core and take the first user`s hands to animate the skeleton.
/// @brief Stores all the possible skeleton settings.
typedef struct SkeletonSettings
{
bool scaleToTarget;
bool useEndPointApproximations;
CollisionType collisionType;
SkeletonTargetType targetType;
SkeletonTargetUserData skeletonTargetUserData;
SkeletonTargetUserIndexData skeletonTargetUserIndexData;
SkeletonTargetAnimationData skeletonTargetAnimationData;
SkeletonTargetGloveData skeletonGloveData;
} SkeletonSettings;
There are several different target types which allow the user to choose how the skeleton is animated. The simplest is the user index, which we explained above, it requires very little knowledge of which gloves or users are available. When changing the target type make sure to match the targetType with the structure of the data, for example the SkeletonTargetType_GloveData enumerator requires the use of the SkeletonTargetGloveData structure.
The user data target allows you to specify a user ID which will enable the skeleton to be animated by the user specified, this does require you to know which users are available. Please keep in mind that the ID is not the same as the index, and you would need to find a certain user's ID in the landscape. The glove data target allows you to specify a glove ID. This ID will be used to find the correct glove and animate your skeleton according to the given glove. If this glove does not exist, the skeleton will not be animated.
To start adding the nodes and the chains to the skeleton setup we need to make it into a Temporary Skeleton by calling the CoreSdk_CreateSkeletonSetup function, which returns an index value which we can use to modify and load the skeleton.
Every skeleton needs a root node to define a point where everything originates from. The ID of the root node should be 0, and the parent of every other node should have a parent id.
In our example we setup our skeleton nodes with the following positions:
Note
These values are only illustrative and just show and example of how you could set up the nodes for a hand skeleton. The actual values will depend on the hand model you are using.
NodeSetup MakeNode(uint32_t p_Id, uint32_t p_ParentId, float p_X, float p_Y, float p_Z, const char* p_Name)
{
NodeSetup t_Node;
NodeSetup_Init(&t_Node);
t_Node.id = p_Id; // unique per node in a skeleton
t_Node.parentID = p_ParentId; // the root has itself (0) as parent
t_Node.type = NodeType_Joint;
t_Node.settings.usedSettings = NodeSettingsFlag_None;
t_Node.transform.position = { p_X, p_Y, p_Z };
std::snprintf(t_Node.name, sizeof(t_Node.name), "%s", p_Name);
return t_Node;
}
// Example joint positions of a hand lying flat on a table, in meters,
// world space. Per finger: 4 joints from knuckle to tip.
// (Illustrative values - replace with your own model's data.)
const float g_FingerX[5] = { 0.025320f, 0.052904f, 0.051287f, 0.049802f, 0.047309f };
const float g_FingerY[5] = { 0.024950f, 0.011181f, 0.000000f, -0.011274f, -0.020145f };
const float g_JointLengths[5][3] = {
{ 0.032742f, 0.028739f, 0.028739f }, // thumb
{ 0.038257f, 0.020884f, 0.018759f }, // index
{ 0.041861f, 0.024766f, 0.019683f }, // middle
{ 0.039736f, 0.023564f, 0.019868f }, // ring
{ 0.033175f, 0.018020f, 0.019129f }, // pinky
};
bool SetupHandNodes(uint32_t p_SetupIndex, Side p_Side)
{
// Root node: id 0, parent 0 (= no parent).
if (CoreSdk_AddNodeToSkeletonSetup(p_SetupIndex, MakeNode(0, 0, 0, 0, 0, "Hand")) != SDKReturnCode_Success)
return false;
// The left hand mirrors the right on the Y axis.
const float t_Mirror = p_Side == Side_Left ? -1.0f : 1.0f;
uint32_t t_NodeId = 1;
for (int t_Finger = 0; t_Finger < 5; ++t_Finger)
{
float t_X = g_FingerX[t_Finger];
const float t_Y = t_Mirror * g_FingerY[t_Finger];
uint32_t t_ParentId = 0;
for (int t_Joint = 0; t_Joint < 4; ++t_Joint)
{
if (CoreSdk_AddNodeToSkeletonSetup(p_SetupIndex,
MakeNode(t_NodeId, t_ParentId, t_X, t_Y, 0.0f, "fingerdigit")) != SDKReturnCode_Success)
return false;
t_ParentId = t_NodeId++;
if (t_Joint < 3) t_X += g_JointLengths[t_Finger][t_Joint];
}
}
return true;
}
We use these positions when creating a NodeSetup for each of the nodes, as seen in the MakeNode function. The resulting structure is then passed into the CoreSdk_AddNodeToSkeletonSetup function which adds it to the Temporary Skeleton.
To have a functional hand skeleton we need to have a Hand chain which contains the ids of all finger chains, and it should have the node ID of a wrist along with which side the hand is from. The handMotion setting in the hand chain specifies if the wrist should move via IMU data or via tracker data, or not at all. Each of the needed chains can be added with the CoreSdk_AddChainToSkeletonSetup function. It is also possible to assign nodes to chains in the Dev Tools, we will explain this in more details during the Temporary Skeletons part of this article.
In our example we add a wrist and all 5 fingers:
bool SetupHandChains(uint32_t p_SetupIndex, Side p_Side)
{
// Chains tell Core which nodes belong to which body part and what
// glove data drives them. First the hand (wrist) chain...
{
ChainSettings t_Settings;
ChainSettings_Init(&t_Settings);
t_Settings.usedSettings = ChainType_Hand;
t_Settings.hand.handMotion = HandMotion_IMU;
t_Settings.hand.fingerChainIdsUsed = 5;
for (int i = 0; i < 5; ++i) t_Settings.hand.fingerChainIds[i] = i + 1;
ChainSetup t_Chain;
ChainSetup_Init(&t_Chain);
t_Chain.id = 0;
t_Chain.type = ChainType_Hand;
t_Chain.dataType = ChainType_Hand;
t_Chain.side = p_Side;
t_Chain.dataIndex = 0;
t_Chain.nodeIdCount = 1;
t_Chain.nodeIds[0] = 0; // the root node
t_Chain.settings = t_Settings;
if (CoreSdk_AddChainToSkeletonSetup(p_SetupIndex, t_Chain) != SDKReturnCode_Success) return false;
}
// ...then one chain per finger, each linking back to the hand chain.
const ChainType t_FingerTypes[5] = {
ChainType_FingerThumb, ChainType_FingerIndex, ChainType_FingerMiddle,
ChainType_FingerRing, ChainType_FingerPinky };
for (int i = 0; i < 5; ++i)
{
ChainSettings t_Settings;
ChainSettings_Init(&t_Settings);
t_Settings.usedSettings = t_FingerTypes[i];
t_Settings.finger.handChainId = 0;
t_Settings.finger.metacarpalBoneId = -1; // no separate metacarpal node in this example
t_Settings.finger.useLeafAtEnd = false;
ChainSetup t_Chain;
ChainSetup_Init(&t_Chain);
t_Chain.id = i + 1;
t_Chain.type = t_FingerTypes[i];
t_Chain.dataType = t_FingerTypes[i];
t_Chain.side = p_Side;
t_Chain.dataIndex = 0;
t_Chain.nodeIdCount = 4;
for (int j = 0; j < 4; ++j) t_Chain.nodeIds[j] = (i * 4) + 1 + j;
t_Chain.settings = t_Settings;
if (CoreSdk_AddChainToSkeletonSetup(p_SetupIndex, t_Chain) != SDKReturnCode_Success) return false;
}
return true;
}
Adding the finger chains goes in a similar process to the hand chain. We need to make sure to add all the node ids and set the handChainId to the same id as the one set in the hand chain, which is 0 in our example. Do not forget to set the correct side for the finger chains as well.
At this point we have a fully functioning Temporary Skeleton. This skeleton can either be loaded into the retargeting system of MANUS Core or sent to the Dev Tools for verification via the CoreSdk_SaveTemporarySkeleton function, which we will get more into in the Temporary Skeletons section of this article.
Retargeted Skeletons
After setting up a Temporary Skeleton via the method explained above the user can call the CoreSdk_LoadSkeleton function. This will load the Temporary Skeleton into MANUS Core's retargeting system and remove it from the list of Temporary Skeletons (because it is no longer temporary at this point). The ID returned from the load function is the one you should keep track of to match the skeleton stream data to the skeleton you loaded in.
It is also possible to unload a skeleton when you no longer need it to be animated. This prevents unnecessary calculations being done for a skeleton that may no longer be in use.
SdkResult SkeletonsRetargeted::Unload(uint32_t p_SkeletonId)
{
const SDKReturnCode t_Result = CoreSdk_UnloadSkeleton(p_SkeletonId);
std::lock_guard<std::mutex> t_Lock(g_Mutex);
g_LoadedIds.erase(std::remove(g_LoadedIds.begin(), g_LoadedIds.end(), p_SkeletonId), g_LoadedIds.end());
g_Latest.erase(p_SkeletonId); // no ghost entry after unloading
return t_Result;
}
In this example, if there are skeletons loaded, we can unload a skeleton by calling CoreSdk_UnloadSkeleton and passing the skeleton id.
When a client application is closed, the session will be closed, and MANUS Core will automatically unload all skeletons that belong to that session.
This can also happen when a connection is lost for any reason, but in such a case you can simply reload the skeletons.
Temporary Skeletons
When defining a skeleton, it cannot be fully animated yet but can be sent to Manus Core's Dev Tools for verification. The Dev Tools will automatically launch and load the temporary skeleton. For detailed usage instructions, refer to the Dev Tools article in our knowledge center.
To set up a Temporary Skeleton, which is similar to a regular skeleton but does not require all chains to be defined, follow these steps. The Dev Tools can send back an updated Temporary Skeleton, which can then be used to create a fully animated skeleton. This functionality is demonstrated in both the Unreal Engine and Unity plugins for animation verification.
If the Temporary Skeleton is not destroyed, it can be easily updated by the Dev Tools. However, if the network connection with MANUS Core is lost, the temporary skeleton must be resent for the plugins to continue working.
For well-defined, typically structured skeletons, you can use the CoreSdk_AllocateChainsForSkeletonSetup function to automatically allocate chains. However, this process can be opaque without visualization, so it is recommended to use the Dev Tools for this purpose. The example client does not use it: it adds its chains explicitly, as shown in the Setup a skeleton section.
Refer to the SkeletonsTemporary::BuildExample function in the sample client for an example of building a setup and sharing it with the Dev Tools — note that it calls CoreSdk_SaveTemporarySkeleton without loading the skeleton, which is what makes the Dev Tools pick it up. Temporary skeletons can also be saved to and loaded from .mskl files for easier handling. Examples of saving and loading .mskl files can be found in the SkeletonsTemporary::SaveToFile and SkeletonsTemporary::LoadFromFile functions; the client defaults these to a ManusTemporarySkeleton folder in your documents directory.
SdkResult SkeletonsTemporary::BuildExample(uint32_t p_SessionId, uint32_t& p_OutIndex)
{
SdkResult t_Result = SkeletonsRetargeted::CreateExampleHandSetup(Side_Left, p_OutIndex);
if (!t_Result.Ok()) return t_Result;
// Saving (without loading) is what shares the setup with DevTools.
t_Result = CoreSdk_SaveTemporarySkeleton(p_OutIndex, p_SessionId, false);
if (t_Result.Ok()) TrackIndex(p_OutIndex);
return t_Result;
}
SdkResult SkeletonsTemporary::SaveToFile(uint32_t p_SkeletonIndex, uint32_t p_SessionId, const std::string& p_FilePath)
{
// Compression must be requested first; it also reports the data size.
uint32_t t_SizeInBytes = 0;
SdkResult t_Result = CoreSdk_CompressTemporarySkeletonAndGetSize(p_SkeletonIndex, p_SessionId, &t_SizeInBytes);
if (!t_Result.Ok()) return t_Result;
std::vector<unsigned char> t_Data(t_SizeInBytes);
t_Result = CoreSdk_GetCompressedTemporarySkeletonData(t_Data.data(), t_SizeInBytes);
if (!t_Result.Ok()) return t_Result;
std::error_code t_Error;
std::filesystem::create_directories(std::filesystem::path(p_FilePath).parent_path(), t_Error);
std::ofstream t_File(p_FilePath, std::ios::binary);
if (!t_File) return SDKReturnCode_InvalidArgument;
t_File.write((const char*)t_Data.data(), t_Data.size());
return SDKReturnCode_Success;
}
SdkResult SkeletonsTemporary::LoadFromFile(uint32_t p_SessionId, const std::string& p_FilePath, uint32_t& p_OutIndex)
{
p_OutIndex = 0;
std::ifstream t_File(p_FilePath, std::ios::binary);
if (!t_File) return SDKReturnCode_InvalidArgument;
std::vector<unsigned char> t_Data(
(std::istreambuf_iterator<char>(t_File)), std::istreambuf_iterator<char>());
if (t_Data.empty()) return SDKReturnCode_InvalidArgument;
// An empty setup acts as the container for the file's data.
SkeletonSetupInfo t_Setup;
SkeletonSetupInfo_Init(&t_Setup);
SdkResult t_Result = CoreSdk_CreateSkeletonSetup(t_Setup, &p_OutIndex);
if (!t_Result.Ok()) return t_Result;
t_Result = CoreSdk_GetTemporarySkeletonFromCompressedData(
p_OutIndex, p_SessionId, t_Data.data(), (uint32_t)t_Data.size());
if (t_Result.Ok()) TrackIndex(p_OutIndex);
return t_Result;
}
-
Create a Temporary Skeleton: Use the
CoreSdk_SaveTemporarySkeletonfunction to save the temporary skeleton to Manus Core. -
Compress the Skeleton Data: Call the
CoreSdk_CompressTemporarySkeletonAndGetSizefunction to get the number of bytes needed to save the data. -
Retrieve the Compressed Data: Use the
CoreSdk_GetCompressedTemporarySkeletonDatafunction to get the actual data into a byte array. -
Save to File: Save the byte array to a file as you would with any normal byte array.
-
Clear (Optional): When you no longer need the temporary skeleton, remove it with
CoreSdk_ClearTemporarySkeleton(seeSkeletonsTemporary::Clear). This step is not required if you plan on using the skeleton.
Clearing all temporary skeletons
CoreSdk_ClearAllTemporarySkeletons (SkeletonsTemporary::ClearAll) removes the temporary skeletons of every connected session, not just your own, so it will disturb other clients working against the same MANUS Core. The example client puts it behind a confirmation for that reason. CoreSdk_GetTemporarySkeletonCountForAllSessions reports how many exist across all sessions.
To load a temporary skeleton from a file:
-
Read the File: Use the example function
SkeletonsTemporary::LoadFromFileto read the file. -
Send to Manus Core: Send the byte array read from the file to Manus Core using the
CoreSdk_GetTemporarySkeletonFromCompressedDatafunction. This will create a new temporary skeleton and return its ID.
Trackers
Manus Core is able to ingest positional and rotation data from various tracking systems (for example ART and SteamVR) and can be used to set up a skeleton for the whole body. For this, they must be assigned to a user and configured to represent a limb or body part location. You may also add your own tracker system and stream the tracker data into MANUS core.
Tracker meta data (such as which user a tracker is assigned to) is part of the landscape, live tracker poses are received via the tracker stream:
void OnTrackerStreamCallback(const TrackerStreamInfo* const p_Info)
{
// The stream announces the count; each tracker is fetched by index.
for (uint32_t i = 0; i < p_Info->trackerCount; ++i)
{
TrackerData t_Data{};
if (CoreSdk_GetTrackerData(i, &t_Data) != SDKReturnCode_Success) continue;
std::lock_guard<std::mutex> t_Lock(g_TrackerMutex);
g_TrackerData[t_Data.trackerId.id] = t_Data;
}
}
The callback stores the latest pose per tracker ID. For normal body tracking the trackers are usually assigned to a user. A screen that wants to display the trackers asks for a copy of the data:
std::vector<TrackerData> Trackers::LatestData()
{
std::lock_guard<std::mutex> t_Lock(g_TrackerMutex);
std::vector<TrackerData> t_Result;
t_Result.reserve(g_TrackerData.size());
for (const auto& t_Entry : g_TrackerData) t_Result.push_back(t_Entry.second);
return t_Result;
}
Using the meta data from the Landscape, you can determine if and where a tracker is assigned. The SDK also offers functions to query trackers per user, such as CoreSdk_GetIdsOfAvailableTrackersForUserIndex.
Adding and removing custom trackers
SdkResult Trackers::SendTestTracker(float p_HeightOffset)
{
TrackerData t_Data;
TrackerData_Init(&t_Data);
FillTrackerId(t_Data.trackerId, "Test Tracker");
t_Data.isHmd = false;
t_Data.trackerType = TrackerType_Unknown;
t_Data.position = { 0.0f, p_HeightOffset, 0.0f };
t_Data.rotation = { 1.0f, 0.0f, 0.0f, 0.0f };
t_Data.quality = TrackingQuality_Trackable;
t_Data.trackedPoint = TrackedPoint_Casing;
return CoreSdk_SendDataForTrackers(&t_Data, 1);
}
In this code example a simple TestTracker is set up and passed into MANUS Core by passing an array of trackers into the CoreSdk_SendDataForTrackers function. As a test, the position is slightly altered every update to show it moving and is visualized in the MANUS Core Dashboard if it is also running. When you make your own trackers make sure every tracker has a unique ID and is trackable.
In this test only one tracker is being sent, but it is advisable for synchronization to send all custom trackers in the same array at the same time.
Assigning Trackers
Trackers can be assigned through use of the CoreSdk_AssignTrackerToUser and CoreSdk_AssignRoleToTracker calls.
The example assigns a tracker to a user and gives it a role (for example the left hand) in one function:
SdkResult Trackers::AssignToUser(const std::string& p_TrackerId, uint32_t p_UserId, TrackerType p_Role)
{
TrackerId t_Id;
FillTrackerId(t_Id, p_TrackerId);
SDKReturnCode t_Result = CoreSdk_AssignTrackerToUser(t_Id, p_UserId);
if (t_Result != SDKReturnCode_Success) return t_Result;
return CoreSdk_AssignRoleToTracker(t_Id, p_Role);
}
Unassigning Trackers
Similarly, trackers can also be unassigned. This is done by assigning them to ID 0.
SdkResult Trackers::Unassign(const std::string& p_TrackerId)
{
// Trackers are unassigned by assigning them to user id 0.
TrackerId t_Id;
FillTrackerId(t_Id, p_TrackerId);
SDKReturnCode t_Result = CoreSdk_AssignTrackerToUser(t_Id, 0);
if (t_Result != SDKReturnCode_Success) return t_Result;
return CoreSdk_AssignRoleToTracker(t_Id, TrackerType_Unknown);
}
Setting Tracker Offset
To set the tracker offset, you can use the CoreSdk_SetTrackerOffset function (wrapped by Trackers::SetOffset). This function is used to set the tracker offset for a specific user. In the example below, the offset maps the left-hand tracker to the glove casing: the entered translation is used, or when the input is left empty, the MANUS Universal Mount's example measurements for a left-hand tracker (0.003 -0.058 -0.043, in meters in the client's coordinate system). Adjust these values to the actual tracker position on your mount.
// Parses "x y z" (meters); empty input falls back to the MANUS
// Universal Mount's example measurements for a left-hand tracker.
void CommitOffsetInput(TrackersState& p_State)
{
SetKeyboardCapture(false);
p_State.mode = TrackersState::Mode::Normal;
p_State.pending = TrackersState::Pending::None;
TrackerOffset t_Offset{};
t_Offset.rotation = { 1.0f, 0.0f, 0.0f, 0.0f };
t_Offset.entryType = TrackerOffsetType_LeftHandTrackerToCasing;
if (p_State.offsetInput.find_first_not_of(" \t") == std::string::npos)
{
t_Offset.translation = { 0.003f, -0.058f, -0.043f };
}
else
{
float t_X = 0.0f, t_Y = 0.0f, t_Z = 0.0f;
if (std::sscanf(p_State.offsetInput.c_str(), "%f %f %f", &t_X, &t_Y, &t_Z) != 3)
{
Status::Error("\"" + p_State.offsetInput + "\" is not 3 numbers - expected \"x y z\" in meters.");
return;
}
t_Offset.translation = { t_X, t_Y, t_Z };
}
const manus::SdkResult t_Result = manus::Trackers::SetOffset(p_State.pendingOffsetUserId, t_Offset);
if (!t_Result.Ok()) Status::Error("Failed to set offset: " + t_Result.Description());
else Status::Success("User " + Id(p_State.pendingOffsetUserId) + ": left-hand tracker offset set to "
+ Vec3String(t_Offset.translation) + ".");
}
Tracking Settings
Various tracking & tracking system settings can be modified.
Tracker timeouts
One such configurable setting is the tracker timeout, which applies to all tracking systems. If no data is received from a tracker within the specified timeout period, it is removed from the landscape and will no longer be used to position any gloves until data from that tracker is received again.
This behavior can be enabled or disabled using CoreSdk_SetTrackerTimeOutEnabled, and the timeout duration (in seconds) can be set using CoreSdk_SetTrackerTimeOut.
To check the current configuration, use CoreSdk_GetTrackerTimeOutEnabled to check whether the timeout is enabled and CoreSdk_GetTrackerTimeOut to retrieve the current timeout value.
SdkResult Trackers::GetTimeout(bool& p_OutEnabled, float& p_OutSeconds)
{
p_OutEnabled = false;
p_OutSeconds = 0.0f;
SDKReturnCode t_Result = CoreSdk_GetTrackerTimeOutEnabled(&p_OutEnabled);
if (t_Result != SDKReturnCode_Success) return t_Result;
return CoreSdk_GetTrackerTimeOut(&p_OutSeconds);
}
SdkResult Trackers::SetTimeoutEnabled(bool p_Enabled)
{
return CoreSdk_SetTrackerTimeOutEnabled(p_Enabled);
}
SdkResult Trackers::SetTimeoutDuration(float p_Seconds)
{
return CoreSdk_SetTrackerTimeOut(p_Seconds);
}
Tracker Systems Settings
Settings specific to certain tracker systems can also be read and modified. This is done through the CoreSdk_GetTrackerSystemsSettings and CoreSdk_SetTrackerSystemsSettings functions.
As a general flow the CoreSdk_GetTrackerSystemsSettings should first be used to get a list of all available tracking systems and their settings.
The desired changes should be made to the settings and then the entire list should be pushed back into the SDK using CoreSdk_SetTrackerSystemsSettings.
As an example, the code snippet below shows how to enable a tracking system.
First, the CoreSdk_GetTrackerSystemsSettings call is used to get all the available tracking system settings.
The settings are stored in the TrackerSystem struct.
Next, the setting of choice is modified. In this case we modify whether the selected tracker system is enabled.
Next, the entire structure is pushed back into the SDK using CoreSdk_SetTrackerSystemsSettings.
SdkResult Trackers::GetSystems(std::vector<TrackerSystem>& p_OutSystems)
{
p_OutSystems.clear();
TrackerSystem t_Systems[MAX_NUM_TRACKER_SYSTEMS];
uint32_t t_Count = 0;
const SDKReturnCode t_Result = CoreSdk_GetTrackerSystemsSettings(t_Systems, &t_Count);
if (t_Result != SDKReturnCode_Success) return t_Result;
p_OutSystems.assign(t_Systems, t_Systems + t_Count);
return t_Result;
}
SdkResult Trackers::SetSystems(const std::vector<TrackerSystem>& p_Systems)
{
// Core expects the full array as obtained from GetSystems.
TrackerSystem t_Systems[MAX_NUM_TRACKER_SYSTEMS]{};
const size_t t_Count = std::min<size_t>(p_Systems.size(), MAX_NUM_TRACKER_SYSTEMS);
std::copy(p_Systems.begin(), p_Systems.begin() + t_Count, t_Systems);
return CoreSdk_SetTrackerSystemsSettings(t_Systems);
}
The Trackers screen of the example client toggles a system with exactly this pattern:
t_State->systems[t_State->selectedSystem].active = !t_State->systems[t_State->selectedSystem].active;
const manus::SdkResult t_Result = manus::Trackers::SetSystems(t_State->systems);
The TrackerSystem has a active value which determines if the tracking system is currently enabled.
It also has a trackerSystemSettings value which contains an array of TrackerSystemSetting.
Tracker system settings can be various types. TrackerSystemSetting has a TrackerSystemSettingType field which determines which type it is.
The Types are:
TrackerSystemSettingInt, which contains an intvalue, aminand amax. Thevalueis clamped to be betweenminandmax.TrackerSystemSettingBool, which contains only a boolvalue.TrackerSystemSettingFile, which contains a stringvaluewhich contains the full path of the file excluding the extension and a stringextensionwhich contains the extension.TrackerSystemSettingIp, which contains a stringvalueof an ip address. For example: "127.0.0.1"
Below is a sample which modifies a selected setting of a selected tracker system.
TrackerSystem& t_System = p_State.systems[p_State.selectedSystem];
if (p_State.selectedSetting >= (int)t_System.currentSettingsCount) return;
TrackerSystemSetting& t_Setting = t_System.trackerSystemSettings[p_State.selectedSetting];
switch (t_Setting.settingtype)
{
case TrackerSystemSettingType_SettingInt:
try { t_Setting.settingInt.value = std::stoi(p_State.input); }
catch (...) { Status::Error("\"" + p_State.input + "\" is not a valid integer."); return; }
break;
case TrackerSystemSettingType_SettingFile:
std::snprintf(t_Setting.settingFile.value, sizeof(t_Setting.settingFile.value), "%s", p_State.input.c_str());
break;
case TrackerSystemSettingType_SettingIp:
std::snprintf(t_Setting.settingIp.value, sizeof(t_Setting.settingIp.value), "%s", p_State.input.c_str());
break;
default:
return;
}
const manus::SdkResult t_Result = manus::Trackers::SetSystems(p_State.systems);
Timecode
A timecode generator is best configured via the MANUS Core Dashboard. However, you might still be interested in the timecode settings in your own client. The timecode configuration (interfaces, frame rate, sync status) is part of the landscape's time member, which the Timecode screen of the example client displays. The sdk/Timecode.cpp module shows how to interpret stream timestamps as timecode:
std::string Timecode::TimecodeString(ManusTimestamp p_Timestamp, bool& p_OutIsTimecode)
{
p_OutIsTimecode = false;
ManusTimestampInfo t_Info{};
if (CoreSdk_GetTimestampInfo(p_Timestamp, &t_Info) != SDKReturnCode_Success) return "";
p_OutIsTimecode = t_Info.timecode;
char t_Buffer[24];
if (t_Info.timecode)
{
// In timecode the fraction is the frame number.
std::snprintf(t_Buffer, sizeof(t_Buffer), "%02u:%02u:%02u:%02u",
t_Info.hour, t_Info.minute, t_Info.second, t_Info.fraction);
}
else
{
// Without timecode the fraction is milliseconds (wall clock, UTC).
std::snprintf(t_Buffer, sizeof(t_Buffer), "%02u:%02u:%02u.%03u",
t_Info.hour, t_Info.minute, t_Info.second, t_Info.fraction);
}
return t_Buffer;
}
If the landscape callback is registered, you can read out the landscape data to retrieve the time landscape with timecode information.
Due to thread safety, the landscape is copied under a mutex and read via LandscapeCache::Snapshot(). The Timecode::FPSName helper translates the TimecodeFPS enumeration into a readable frame rate.
For more information on timecode, please consult your timecode device manual.
Gestures
MANUS Core has a built-in gesture system. Data from this gesture system can be accessed via the Gesture Stream. In our example in the function OnGestureStreamCallback you can see how to receive the data:
void OnGestureStreamCallback(const GestureStreamInfo* const p_Info)
{
FeedTimestamp(p_Info->publishTime);
for (uint32_t i = 0; i < p_Info->gestureProbabilitiesCount; ++i)
{
// The stream announces how many probability sets are available;
// each set is fetched in chunks of MAX_GESTURE_DATA_CHUNK_SIZE.
GestureProbabilities t_Chunk{};
if (CoreSdk_GetGestureStreamData(i, 0, &t_Chunk) != SDKReturnCode_Success) continue;
if (t_Chunk.isUserID) continue; // sets can belong to users too; we buffer per glove
const uint32_t t_GloveId = t_Chunk.id;
GestureStream::GloveGestures t_Data;
t_Data.totalCount = t_Chunk.totalGestureCount;
t_Data.probabilities.reserve(t_Chunk.totalGestureCount);
uint32_t t_Read = 0;
while (true)
{
for (uint32_t j = 0; j < t_Chunk.gestureCount; ++j)
t_Data.probabilities.push_back(t_Chunk.gestureData[j]);
t_Read += t_Chunk.gestureCount;
if (t_Read >= t_Data.totalCount || t_Chunk.gestureCount == 0) break;
if (CoreSdk_GetGestureStreamData(i, t_Read, &t_Chunk) != SDKReturnCode_Success) break;
}
std::lock_guard<std::mutex> t_Lock(g_GestureMutex);
g_GloveGestures[t_GloveId] = std::move(t_Data);
}
}
The stream info gives you the number of gestures available per glove. Using the CoreSdk_GetGestureStreamData function you can get the specific gesture information for a given glove. Due to the amount of potential gesture data in the future, it is advisable to call this function in the callback to gather all the gesture data. This way it becomes impossible for the data to be modified by the SDK during this function call. In our example we gather the gesture data for every glove; the Gestures screen then matches the gesture IDs to their names from the landscape and displays them:
manus::GestureStream::GloveGestures t_Gestures;
if (!manus::GestureStream::LatestForGlove(p_GloveId, t_Gestures))
return vbox({
text("No gesture data received for glove " + Id(p_GloveId) + " yet.") | dim,
RenderAssignmentHint(p_GloveId),
}) | center;
// Gesture names come from the landscape.
std::map<uint32_t, std::string> t_Names;
for (const GestureLandscapeData& t_Gesture : manus::LandscapeCache::Gestures())
t_Names[t_Gesture.id] = t_Gesture.name;
// Highest confidence first.
std::vector<GestureProbability> t_Sorted = t_Gestures.probabilities;
std::sort(t_Sorted.begin(), t_Sorted.end(),
[](const GestureProbability& a, const GestureProbability& b) { return a.percent > b.percent; });
The gesture data you receive contains ID and normalized percentage numbers (from 0.0 to 1.0). The gesture ID can be matched to the gesture name shown in the landscape data for gestures, which will allow you to see what gesture it is.
Currently it is not supported to add custom gestures to MANUS Core , when this does become possible it is best to simply remember what IDs are assigned to which gesture. In predefined gestures, the IDs will remain the same throughout the application runtime, so you will only need to match the gesture's name to its ID once if you wish to use those names for identification.
Calibrating gloves
To calibrate a glove, a specific series of steps must be followed. The number of steps can vary between different glove models. To determine how many steps are required for a specific glove, you can use the CoreSdk_GloveCalibrationGetNumberOfSteps function, passing an argument of type GloveCalibrationArgs.
For detailed information about each calibration step, the CoreSdk_GloveCalibrationGetStepData function can be used. This method takes a GloveCalibrationStepArgs parameter and returns a GloveCalibrationStepData object. The returned data includes an index, title, description, and duration. If the duration is an estimated value, it will be negative.
To start the calibration process, call the CoreSdk_GloveCalibrationStart method, and to stop it, use the CoreSdk_GloveCalibrationStop method. Each calibration step must be performed in sequential order using the CoreSdk_GloveCalibrationStartStep method, the GloveCalibrationStepArgs contains information on which step to start. After all steps are completed, call CoreSdk_GloveCalibrationFinish to save and apply the calibration.
sdk/Calibration.cpp wraps each of these calls. Note that CoreSdk_GloveCalibrationStartStep blocks while the step is being recorded, the example client therefore runs the calibration wizard on a separate thread.
SdkResult Calibration::GetNumberOfSteps(uint32_t p_GloveId, uint32_t& p_OutCount)
{
p_OutCount = 0;
GloveCalibrationArgs t_Args{ p_GloveId };
return CoreSdk_GloveCalibrationGetNumberOfSteps(t_Args, &p_OutCount);
}
SdkResult Calibration::GetStep(uint32_t p_GloveId, uint32_t p_StepIndex, Step& p_OutStep)
{
GloveCalibrationStepArgs t_Args{ p_GloveId, p_StepIndex };
GloveCalibrationStepData t_Data;
GloveCalibrationStepData_Init(&t_Data);
const SDKReturnCode t_Result = CoreSdk_GloveCalibrationGetStepData(t_Args, &t_Data);
if (t_Result != SDKReturnCode_Success) return t_Result;
p_OutStep.index = t_Data.index;
p_OutStep.title = t_Data.title;
p_OutStep.description = t_Data.description;
p_OutStep.time = t_Data.time;
return t_Result;
}
SdkResult Calibration::Start(uint32_t p_GloveId, bool& p_OutAccepted)
{
p_OutAccepted = false;
GloveCalibrationArgs t_Args{ p_GloveId };
return CoreSdk_GloveCalibrationStart(t_Args, &p_OutAccepted);
}
SdkResult Calibration::ExecuteStep(uint32_t p_GloveId, uint32_t p_StepIndex, bool& p_OutSucceeded)
{
p_OutSucceeded = false;
GloveCalibrationStepArgs t_Args{ p_GloveId, p_StepIndex };
return CoreSdk_GloveCalibrationStartStep(t_Args, &p_OutSucceeded);
}
SdkResult Calibration::Finish(uint32_t p_GloveId, bool& p_OutAccepted)
{
p_OutAccepted = false;
GloveCalibrationArgs t_Args{ p_GloveId };
return CoreSdk_GloveCalibrationFinish(t_Args, &p_OutAccepted);
}
SdkResult Calibration::Cancel(uint32_t p_GloveId, bool& p_OutAccepted)
{
p_OutAccepted = false;
GloveCalibrationArgs t_Args{ p_GloveId };
return CoreSdk_GloveCalibrationStop(t_Args, &p_OutAccepted);
}
Saving / Loading glove calibrations
Although calibrations are saved automatically in the settings file, sometimes it can be useful to manually manage calibrations instead. In this section we outline the various ways in which glove calibrations can be saved and loaded. This can be useful when wanting to transfer glove calibrations between different machines or between Remote and Integrated modes.
Limitations
It is important to note that glove calibrations can only be saved for the Metaglove and Metaglove Pro series gloves. Metaglove and Metaglove Pro calibrations are also fundamentally different. A Metaglove glove calibration cannot be used for a Metaglove Pro glove and vice versa.
Also be wary the management of calibrations works slightly differently depending on if Auto Assignment is enabled or not. As a rule of thumb, when Auto Assignment is enabled calibrations are managed on a per glove basis. When Auto Assignment is disabled calibrations are managed on a per user basis. This means gloves will use the calibration of the user they are assigned to. For more information on user management see Users.
Saving glove Calibrations
To save glove calibrations, use the CoreSdk_GetGloveCalibrationSize and CoreSdk_GetGloveCalibration functions. The CoreSdk_GetGloveCalibrationSize must always be called first. It stores the calibration data in a byte array and returns its size. Next the CoreSdk_GetGloveCalibration can be called to get the byte array. It can then be saved to a file. The following code snippet demonstrates how to save calibrations to a file. The file is saved in the user's documents directory under a folder named manus-calibrations.
SdkResult Calibration::SaveToFile(uint32_t p_GloveId, const std::string& p_FilePath)
{
// The size call also PREPARES the calibration data for retrieval.
uint32_t t_Size = 0;
SDKReturnCode t_Result = CoreSdk_GetGloveCalibrationSize(p_GloveId, &t_Size);
if (t_Result != SDKReturnCode_Success) return t_Result;
std::vector<unsigned char> t_Data(t_Size);
t_Result = CoreSdk_GetGloveCalibration(t_Data.data(), t_Size);
if (t_Result != SDKReturnCode_Success) return t_Result;
std::error_code t_Error;
std::filesystem::create_directories(std::filesystem::path(p_FilePath).parent_path(), t_Error);
std::ofstream t_File(p_FilePath, std::ios::binary);
if (!t_File) return SDKReturnCode_InvalidArgument;
t_File.write((const char*)t_Data.data(), t_Data.size());
return SDKReturnCode_Success;
}
The CoreSdk_GetGloveCalibrationSize function is used by specifying the ID of the glove to save the calibration for. Alternatively, it is also possible to specify a user and calibration type instead by using the CoreSdk_GetGloveCalibrationSizeForUser function, for example to save the left Metaglove Pro calibration of a specific user.
Calibrations can be loaded for a specific glove by loading an .mcal file and passing it to the CoreSdk_SetGloveCalibration function. When auto assignment is disabled, be sure to assign the glove to a user before calling this function. Note that in this situation the calibration will actually be set for the user.
SdkResult Calibration::LoadFromFile(uint32_t p_GloveId, const std::string& p_FilePath,
SetGloveCalibrationReturnCode& p_OutResult)
{
p_OutResult = SetGloveCalibrationReturnCode_Error;
std::ifstream t_File(p_FilePath, std::ios::binary);
if (!t_File) return SDKReturnCode_InvalidArgument;
std::vector<unsigned char> t_Data(
(std::istreambuf_iterator<char>(t_File)), std::istreambuf_iterator<char>());
if (t_Data.empty()) return SDKReturnCode_InvalidArgument;
return CoreSdk_SetGloveCalibration(
p_GloveId, t_Data.data(), (uint32_t)t_Data.size(), &p_OutResult);
}
Alternatively, calibrations can be loaded for a specific user. This can be done through loading a file and passing it to the CoreSdk_SetGloveCalibrationForUser function. The calibration will automatically be applied to the correct side. Take note that users internally have a separate Metaglove and Metaglove Pro calibration. Loading a Metaglove calibration for a user will not affect connected Metaglove Pro gloves and vice versa.
Pairing / Unpairing
To pair or unpair a glove, you can use the CoreSdk_PairGlove or CoreSdk_UnpairGlove functions. Both functions require the glove’s ID and a pointer to a boolean value, which will indicate whether the operation was successful. A true value means the glove was successfully paired or unpaired.
When pairing with CoreSdk_PairGlove, the SDK will automatically use the first available dongle. To pair the glove with a specific dongle, use CoreSdk_PairGloveToDongle and pass the ID of the preferred dongle. sdk/Devices.cpp wraps both variants:
SdkResult Devices::PairGlove(uint32_t p_GloveId, bool& p_OutAccepted)
{
p_OutAccepted = false;
return CoreSdk_PairGlove(p_GloveId, &p_OutAccepted);
}
SdkResult Devices::PairGloveToDongle(uint32_t p_GloveId, uint32_t p_DongleId, bool& p_OutAccepted)
{
p_OutAccepted = false;
return CoreSdk_PairGloveToDongle(p_GloveId, p_DongleId, &p_OutAccepted);
}
When pairing to a specific dongle, the example client looks in the landscape for a dongle that still has a free slot for the glove's side:
uint32_t Devices::FindDongleWithFreeSlot(Side p_Side)
{
for (const DongleLandscapeData& t_Dongle : LandscapeCache::DonglesSortedById())
{
// Skip wired gloves acting as their own dongle ("Glongle").
if (t_Dongle.classType != DeviceClassType_Dongle) continue;
const uint32_t t_SlotGlove = p_Side == Side_Left ? t_Dongle.leftGloveID : t_Dongle.rightGloveID;
if (t_SlotGlove == 0) return t_Dongle.id;
}
return 0;
}
Unpairing works the same way, using the ID of a paired glove:
SdkResult Devices::UnpairGlove(uint32_t p_GloveId, bool& p_OutAccepted)
{
p_OutAccepted = false;
return CoreSdk_UnpairGlove(p_GloveId, &p_OutAccepted);
}
Users
Our User System is used to group gloves and dongles by user.
When using Core Integrated, by default users are automatically created as needed when new gloves and dongles are connected. It's however also possible to manually control this using the functions outlined below.
Note
For manual control the Auto-Assignment system should be disabled using CoreSdk_SetAutoUserAssignment(false). Its current state can be read back with CoreSdk_GetAutoUserAssignment — the Users screen of the example client uses both to offer it as a toggle.
When using Core Remote, AutoUserAssignment is always disabled, the manual user system is available in both modes.
Assignment is not only bookkeeping: MANUS Core only produces skeleton, ergonomics and gesture data for gloves that are assigned to a user. Raw device data flows regardless of assignment. If a connected glove produces no data, an unassigned glove is the first thing to check.
The CoreSdk_AddUser function is used to create new users. The function accepts a char array for the name of the user and outputs the ID that was assigned to the user.
In Core Integrated, if an empty username is provided the user will receive a default name of "user" + userID. In Core Remote, no default name is assigned so it is recommended to assign a name manually during creation.
SdkResult Users::Add(const std::string& p_Name, uint32_t& p_OutId)
{
p_OutId = 0;
// The SDK header takes a non-const char*; it does not modify the string.
return CoreSdk_AddUser(const_cast<char*>(p_Name.c_str()), &p_OutId);
}
The CoreSdk_RemoveUser function is used to delete users. The function accepts the ID of the user to be deleted. These User IDs can be discovered using the Landscape.
SdkResult Users::Remove(uint32_t p_UserId)
{
return CoreSdk_RemoveUser(p_UserId);
}
The CoreSdk_AssignGloveToUser function is used to assign gloves to users. It accepts the ID of the user, the ID of the glove and the side of the glove as parameters. Unassigned gloves and available users can be found in the Landscape.
SdkResult Users::AssignGlove(uint32_t p_UserId, uint32_t p_GloveId, Side p_Side)
{
return CoreSdk_AssignGloveToUser(p_UserId, p_GloveId, p_Side);
}
The CoreSdk_AssignGloveToUser can also be used to unassign gloves. This is done by passing 0 as Glove ID parameter in the function.
To find out which user a glove is currently assigned to, the users in the Landscape can be searched for the glove's ID:
bool Users::TryFindUserForGlove(const Landscape& p_Landscape, uint32_t p_GloveId, UserLandscapeData& p_OutUser)
{
if (p_GloveId == 0) return false;
for (uint32_t i = 0; i < p_Landscape.users.userCount; ++i)
{
const UserLandscapeData& t_User = p_Landscape.users.users[i];
if (t_User.leftGloveID == p_GloveId || t_User.rightGloveID == p_GloveId)
{
p_OutUser = t_User;
return true;
}
}
return false;
}
Just like gloves, dongles can be assigned and unassigned to users through the use of the CoreSdk_AssignDongleToUser function.
The function accepts the User ID and Dongle ID as parameters. If you wish to unassign a dongle from a user, use a dongle ID of 0.
SdkResult Users::AssignDongle(uint32_t p_UserId, uint32_t p_DongleId)
{
return CoreSdk_AssignDongleToUser(p_UserId, p_DongleId);
}
Users can be moved through use of the CoreSdk_MoveUserUp and CoreSdk_MoveUserDown functions. The ordering of users is relevant for certain applications such as OpenXR in which we only use the data of the first user in the list.
The functions accept the ID of the user as parameter.
SdkResult Users::MoveUp(uint32_t p_UserId)
{
return CoreSdk_MoveUserUp(p_UserId);
}
SdkResult Users::MoveDown(uint32_t p_UserId)
{
return CoreSdk_MoveUserDown(p_UserId);
}
Renaming users can be done through the CoreSdk_SetUserName function. This function accepts the ID of the user to rename and the new name as parameters.
SdkResult Users::Rename(uint32_t p_UserId, const std::string& p_Name)
{
return CoreSdk_SetUserName(p_UserId, const_cast<char*>(p_Name.c_str()));
}
Raw sensor data
The raw sensor data is accessible through the RawDeviceDataStream and works similarly to the other device data streams. It sends out a structure that contains the IMU rotation and the sensor position and rotations for each of the finger's sensors (5) per glove device.
The order is as follows:
- thumb (0)
- index (1)
- middle (2)
- ring (3)
- pinky (4)
The data is in the specified coordinate system's format. The sensors point outwards from the source in alignment with the "View" axis. When bending the fingers, they rotate around the "Side" axis. They point upwards aligned with the "up" axis. For more information on this coordinate systems, see the Coordinate System section.
The position of the sensors are relative to the magnetic coils inside the casing on the back of the hand. This means that the offsets from sensor to joints have to be taken into consideration when interpreting the data. See the diagram below for the exact positioning of the coils inside the casings. The sensor positions are measured from finger sensor coil relative to the main coil in the casing.
Note
The raw sensor positions only stream out for MANUS Metaglove Pro gloves, and only when the license has the raw feature enabled. For more information, please contact support at support@manus-meta.com.
void OnRawDeviceDataStreamCallback(const RawDeviceDataInfo* const p_Info)
{
FeedTimestamp(p_Info->publishTime);
for (uint32_t i = 0; i < p_Info->rawDeviceDataCount; ++i)
{
RawDeviceData t_Data{};
if (CoreSdk_GetRawDeviceData(i, &t_Data) != SDKReturnCode_Success) continue;
std::lock_guard<std::mutex> t_Lock(g_RawDeviceMutex);
g_RawDeviceData[t_Data.id] = t_Data;
}
}
The stored data can then be fetched per glove device:
bool RawDeviceStream::LatestForDevice(uint32_t p_DeviceId, RawDeviceData& p_OutData)
{
std::lock_guard<std::mutex> t_Lock(g_RawDeviceMutex);
const auto t_Found = g_RawDeviceData.find(p_DeviceId);
if (t_Found == g_RawDeviceData.end()) return false;
p_OutData = t_Found->second;
return true;
}