Optitrack live mocap streaming to Unreal

This short guide shows how to set up the Optitrack system to stream out data to the Unreal engine.

I’m using Unreal 4.212 with the Motive 2.1.1 and Unreal Engine plugin 1.21 (actually, I have also used Unreal 4.19 and the 4.19 plugin – just make sure the plugin and Unreal versions match.) The later versions of the Optitrack plugin don’t support Oculus headset tracking

First make sure you can ping the other computer and if you are having problems with network discovery see here

SETTING UP MOTIVE TO STREAM DATA::
– this can be either a prerecorded file, a live performer or a VR user

Note the name of the skeleton you want to stream e.g Ruth

Heres my streaming settings:

Optitrack Streaming Settings

Make sure the Asset Settings are like so: Bones on, Segments off.

Asset Properties

::SETTING UP UNREAL to RECEIVE DATA::

Add the Optitrack plugin to the Unreal engine – put it in the plugins directory where the engine is installed.

In Unreal Enable the plugins = Optitrack NatNet (and Optitrack- OculusVR if you need headset tracking – in this case make sure to disable all the other VR plugins eg Oculus Steam etc.)

Import your character mesh and set up like the tute…

Drop an Optitrack Client Origin into the level – this will be the zero point of everything which corresponds to the centre of the Optitrack tracking space.

Select it in and under Optitrack set the server and client e.g 127.1.0.0.1 if it’s testing on the motive local machine, or automatic if its on 2 different machines.

Set streaming to Multicast if using Wifi

Set the HMD RigidBody Id to the Id of the HMD from Motive and the orientation to +X Forward– this is for the headset tracking

here are my settings for the older version of the plugin- like I said local loopback so the server is Multicast 

Optitrack Client Origin Settings in Unreal.

NB really important to set the hmd Orientation to +X forward like the diagram above, else audience character is 90 rotated. The default is Z forward doh!! Also don’t forget to set the HMD Rigid Body Id in Motive to the same as you set above or vice versa.

Also if you recalibrate (the audience member ) from markers in Motive then the HDM ID gets reset to 9999. You can tell if the HMD has got the wrong settings because your audience body is in the wrong place. If you are stuck in the floor with the body above you in VR check for this too. And when compiling the Unreal project remember to set Start in VR in project settings

(From version 1.21 of the plugin which is for Unreal 4.21 there is a super handy auto-connect feature that scans for ‘and connects to Motive automatically (connection settings do not need to be set beforehand). When this feature is on, if the plugin hasn’t received a frame in a few seconds, it’ll attempt to search for and connect to Motive again. Users can open/close Motive or start/stop streaming without restarting their Unreal experience. This feature is NOT supported on Android.’)

::SET UP THE CHARACTER::

In Motionbuilder make sure that your character mesh is not parented to a bone in the hierarchy – unparent it – otherwise, the Optitrack Unreal plugin hates it and crashes.

My mesh (blue nodes) is NOT parented to the skeleton (white nodes)

I’ve been exporting a T-pose take of my performer and using a zeroed out version of the skeleton, as the start and editing my character mesh to fit the size of the performers skeleton (see this post here) as the unreal plugin only gets bone translations, it won’t do retargeting so if the characters are different sizes to performers, there may well be a problem.

Import the mesh into Unreal – open the mesh, under Asset Details look for Miscellaneous and Tick the following:
– Convert Scene
– Force Front XAxis
– Convert Scene Unit

R-Click in the content browser and Create an Animation Blueprint, choose OptitrackAnimInstance and then in Target Skeleton underneath choose<YourSkeletonName>

Set up the Animation blueprint by adding these special Optitrack nodes.  (Note that pulling a wire from the empty Source Skeleton Streaming Asset Name dot and usign Promote to Variable does not work, you need to get the correct variable.) Now save it.

Select the Optitrack Skeleton node In the details panel hit Auto Fill Bone Mapping – in the bone list check and possibly set the correct bone names (***N.B. using bones from Motive, it still managed to get some of them wrong so double-double check this list out)

Recompile and Save.

Drop the Anim BP character into the level, under Details Add component, OptitrackSkeleton and set the Source Skeleton name to your skeleton that’s gonna stream from Motive (and the name is case sensitive) and set the Streaming Client Origin to the  Optitrack Client Origin.

in the 3d window, Show /Developer / Bones may be helpful here. . . . . . . .

(I’ve been having a crash where Unreal borks – the seeming fix is the startup order. So Stop Motive playing, Play Unreal, Play Motive)

and in the OptitrackClient Origin / Details there is a setting for DrawDebug Skeleton– you can see here if the mocap is coming in – you see a skel if your character won’t move. Mine took a bit of clicking and recompiling to get it moving.

White Skel is the Optitrack mocap data streaming in. Red Skel is my character

Something wrong here… so this is in the Mesh import SettingsForce Front X-Axis

Now it’s fixed like this – you can’t see the white skel of the Optitrack data as it’s hidden by the character skel :

 

here’s a video

Plugin here:

https://optitrack.com/downloads/plugins.html

Official Tute here:

https://v21.wiki.optitrack.com/index.php?title=Unreal_Engine_Skeleton_Streaming

There is also info about the Rigid body skeleton streaming for VR here

https://v21.wiki.optitrack.com/index.php?title=Rigid_Body_Skeleton_Markerset