Optitrack live mocap streaming to Unreal

I’m using Unreal 4.20.3 with the Motive 2.1.1 and Unreal Engine plugin 1.2. (actually, I also used Unreal 4.19 and the 4.19 plugin – make sure plugin and Unreal version match up.)

Make sure you can ping the other computer and if you are having problems with network discovery see here

SETTING UP MOTIVE TO STREAM DATA::
– this can be either a prerecorded file, a live performer or a VR user

Note the name of the skeleton you want to stream e.g Ruth

Heres my streaming settings: (seems to hate it if Transmission type is Multicast.)

Motive Streaming Settings

Make sure the Asset Settings are like so: Bones on, Segments off.

Asset Properties

::SETTING UP UNREAL to RECEIVE DATA

Add the Optitrack plugin to the Unreal engine – put it in the plugins directory where the engine is installed.

In Unreal Enable the plugins = Optitrack NatNet and Optitrack- OculusVR. Disable all the other Vr plugins eg Oculus Steam etc.

Import your character mesh and set up like the tute…

Drop an Optitrack Client Origin into the level – this will be the zero point of everything.

Select it in and under Optitrack set the server and client e.g 127.1.0.0.1 if it’s testing on the motive local machine, or automatic if its on 2 different machines.

Set streaming to Multicast if using Wifi

Set the HMD RigidBody Id to the Id of the HMD from Motive and the orientation to +X .

here are my settings for the older version of the plugin- like I said local loopback so the server is Multicast 

 

From version 1.21 of the plugin which is for Unreal 4.21 there is a super handy auto-connect feature that scans for ‘and connects to Motive automatically (connection settings do not need to be set beforehand). When this feature is on, if the plugin hasn’t received a frame in a few seconds, it’ll attempt to search for and connect to Motive again. Users can open/close Motive or start/stop streaming without restarting their Unreal experience. This feature is NOT supported on Android.’

Auto Connect

SET UP THE CHARACTER::

In Motionbuilder make sure that your character mesh is not parented to a bone in the hierarchy – unparent it – otherwise, the Optitrack Unreal plugin hates it and crashes.

My mesh (blue nodes) is NOT parented to the skeleton (white nodes)

I’ve been exporting a T-pose take of my performer and using a zeroed out version of the skeleton, as the start and editing my character mesh to fit the size of the performers skeleton (see this post here) as the unreal plugin only gets bone translations, it won’t do retargeting so if the characters are different sizes to performers, there may well be a problem.

Import the mesh into Unreal – open the mesh, under Asset Details look for Miscellaneous and Tick the following:
– Convert Scene
– Force Front XAxis
– Convert Scene Unit

R-Click in the content browser and Create an Animation Blueprint, choose OptitrackAnimInstance and then in Target Skeleton underneath choose<YourSkeletonName>

 

Set up the Animation blueprint by adding these special Optitrack nodes. Then save it.

Select the Optitrack Skeleton node In the details panel hit Auto Fill Bone Mapping – in the list check and possible set the correct bone names (***N.B. using bones from Motive, it still managed to get some of them wrong so double-double check this list out)

Recompile and Save.

Drop the Anim BP character into the level then under Details Add component, OptitrackSkeleton and set the  Source Skeleton name to your skeleton (that’s gonna stream) in Motive and set the Streaming Client Origin to the  Optitrack Client Origin.

in the 3d window, Show /Developer / Bones may be helpful here. . . . . . . .

(I’ve been having a crash where Unreal borks – the seeming fix is the startup order. So Stop Motive playing, Play Unreal, Play Motive)

and in the OptitrackClient Origin / Details there is a setting for DrawDebug Skeleton– you can see here if the mocap is coming in – you see a skel if your character won’t move. Mine took a bit of clicking and recompiling to get it moving.

White Skel is the Optitrack mocap data streaming in. Red Skel is my character

something wrong here… so this is in the Mesh import SettingsForce Front X-Axis

Now it’s fixed like this – you can’t see the white skel of the Optitrack data as it’s hidden by the character skel :

Official Tute here:

https://v21.wiki.optitrack.com/index.php?title=Unreal_Engine_Skeleton_Streaming

There is also info about the Rigid body skeleton streaming for VR here

https://v21.wiki.optitrack.com/index.php?title=Rigid_Body_Skeleton_Markerset