MRTK3 Setup Guide

This sample scene demonstrates how to integrate MRTK3 into your Unity project. For more information on MRTK, visit the MRTK3 documentation.

Setup a New Unity Project with MRTK3 and Snapdragon Spaces on Windows

This guide leads through the steps for how to setup and build a brand new MRTK3 Project for Snapdragon Spaces.

NOTE

Please avoid components that utilize the virtual keyboard and web browser, as those are not supported at this time and could cause unexpected behavior.

Unity Project Setup

Start with the Unity Setup Guide from Prerequisites down to Enable Spaces Features.

Once set up, keep your Unity Project open and take note of the file path.

Project Configuration for MRTK

• Install MRTK requirements for MRTK3:
  • Visual Studio 2019 Community edition or greater
  • Mixed Reality Feature Tool is required to access the MRTK3 packages. This is currently only available on Windows. Download, but do not run yet.
  • You will need .Net 0.5+ Runtime which is included in Visual Studio 16.11.14+

Import the MRTK3 packages

• Open the Mixed Reality Feature Tool (MixedRealityFeatureTool.exe)

• Wait for it to load up, then click Start.
• Set the Project Path to the file path of the Unity Project.
• Click Discover Features.
  • Click Select All for the MRTK3 line item.
  • Click Get Features. Some recommended dependencies may be displayed. If so, select those too. Click Import when done to download files and update the Unity Project manifest.
  • When finished, optionally close the Feature Tool, or keep it open to make continued adjustments as needed.

Return to Unity to complete package import

• Switch back to the Unity Project and wait for any importing to occur as a result of the new features being added by the Mixed Reality Feature Tool.
  • If prompted, select I Made a Backup, Go Ahead! regarding the XR Interaction Layer Mask.
  • If prompted to restart, choose Yes.

Scene Setup

• Create a new Scene. This scene will be used to set up the camera and input from scratch.

Modify the Camera for Snapdragon Spaces

• Delete the Main Camera from the Scene Hierarchy.
• Find the MRTK XR Rig Prefab and drag it into the Scene, located at: Packages/com.microsoft.mrtk.input/Assets/Prefabs/MRTK XR Rig.prefab

Configure for Snapdragon Spaces Controller Input

As stated above, this guide is only for using the Companion Controller as device input. To set up for Companion Controller, follow the steps below.

• On MRTK XR Rig (top-level GameObject):
  • In the Input Action Manager Component, add a second element to the Action Assets dropdown and set it to: Input Actions located at: Samples/Snapdragon Spaces/[version ID]/Core Samples/Shared Assets/Input Actions. This will map the input actions to the ones set up from the Snapdragon Spaces Core Samples, allowing for proper connection to the Companion Controller.

• Now add the Device Pointer Prefab located at: Samples/Snapdragon Spaces/[version ID]/Core Samples/Shared Assets/Prefabs/Interaction/
  • In the XR Interactor Line Visual Component, change the Line Width to 0.005.

Finally, set the MRTK Profile.

• Create a copy of the MRTKProfile found in Packages > MRTK Core Definitions > Configuration > Default Profiles and drop it into the projects Assets directory.

• Open Edit > Project Settings > MRTK3.
• Drag the copy into the Profile section under the Android tab.
• Uncheck the MRTK Hands Aggregator Subsystem.
• Uncheck the Subsystem for Hand Synthesis.

Optional: Add Interactive Components to test out MRTK functionality

Selectable Object

• Add a 3D Cube to the scene.
  • Set the Position to (0, 0, 1.5)
  • Rotation to (45, 45, 45)
  • Scale to (0.5, 0.5, 0.5)
• Add Component ObjectManipulator to allow grabbing from the laser pointer.

MRTK Button Group

• Test out the MRTK Prefab Interaction components by adding a ButtonGroup_32x32mm_H3 prefab into the Scene.
• This may prompt you to install TextMeshPro. If so, select Import TMP Essentials.
• Select the ButtonGroup in the Hierarchy and in the Inspector, set the Transform Position to 0,0,0.5. Press Play to process scripts and visualize the button faces, then Stop to return to Editing.
• Expand the ButtonGroup to find the three PressableButton_32x32mm_IconAndText objects and expand them one at a time.

• For the first one, select the top-level button object, find the Pressable Button component in the Inspector, click + for OnClicked(), set it to the Cube in the Scene, and set the action to GameObject.SetActive with the checkbox unchecked.
  • Under the child CompressableButtonVisuals:
    • Find the UX.Button.Icon.Char child object and change the SpriteRenderer's Sprite to an empty circle.
• For the second one, select the top-level button object, find the Pressable Button component in the Inspector, click + for OnClicked(), set it to the Cube in the Scene, and set the action to GameObject.SetActive with the checkbox checked.
  • Under the child CompressableButtonVisuals:
    • Find the UX.Button.Icon.Charchild object and change the SpriteRenderer's Sprite to a filled circle.
    • Find the TextMeshPro child and set the Text Input to Show.
• For the third button, you will first need to create a new Script and add it to the Cube. The script should have a publicly accessible function such as

    ...
    public void Quit() {
        Application.Quit();
    }
    ...

• Still on the third button:
  • Select the top-level button object, find the Pressable Button component in the Inspector, click + for OnClicked(), set it to the Cube in the Scene, and set the action to Quit() from the new script.
    • Under the child CompressableButtonVisuals:
      • Find the UX.Button.Icon.Char child object and change the SpriteRenderer's Sprite to a Quite button.
      • Find the TextMeshPro child and set the Text Input to Show.

Testing to device

• Configure the Build Settings and Player Settings to build the app.
• Build the app from the Build Settings window and save your app locally.
• Connect your Android Device and approve USB Debugging if prompted.
• Install the apk to your device.
• Long-press the App Icon, tap App Info, and approve all of the Permissions.
• Plug in the glasses, and once fully loaded, run the app.

---

MRTK Samples

For additional insight into the available MRTK3 user interface components, it is recommended to download the MRTK3 Samples and configure them for Snapdragon Spaces. Follow the steps below to do this.

Get the MRTK3 Samples

• Clone the repo or download the Unity Project samples at: https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/tree/main/UnityProjects
• Extract or Clone this repo in a location with a short filename. This repo has many folders with long names, which can result in file names becoming too long and causing issues with Unity.
• Do not move any of the files in this repo, as there are many dependencies and changing a file’s position will cause issues. If you need to move the repo, move the entire repo altogether.

Open Project and integrate the Snapdragon Spaces:

• Add MRTKDevTemplate from the UnityProjects folder in the Unity Hub.
• Open the Unity Project.
• Import the Snapdragon Spaces package as in the Unity Setup Guide.
• Select a Scene to open. Recommended Scene is HandInteractionExamples.
• Configure the Scene to modify the camera and input as above.
• Build and deploy to device.

Hand Tracking

Hand Tracking Plug-in Setup

Hand tracking is supported using the Mixed Reality OpenXR Plug-in. To get the Plug-in, use the Mixed Reality Feature Tool (see above) and select the Mixed Reality OpenXR Plug-in from the Platform Support section.

Click through to complete the process of updating the project, then return to the Unity project.

Unity Configuration

In Unity, there are two areas to update.

MRTK Settings

The first is the MRTK Settings. Make sure the MRTK Hands Aggregator Subsystem and the Subsystem of OpenXR Hands API checkboxes are both enabled.

Additionally, it is recommended to experiment with calibration of the Pinch Closed Threshold setting, which is set to default at 0.25. Developers have found some success with adjusting that threshold to 0.45 or 0.5 (as shown).

OpenXR Settings

The second area to update is the OpenXR Settings. Notice that there may be two Hand Tracking features available. To avoid conflicts, only enable the Microsoft Hand Tracking checkbox, and disable any other Hand Tracking checkboxes. This is the one with a question mark next to it, and can be validated as the Microsoft version by clicking on the Settings icon to see the Author.

Hand Visualization Settings

In addition to the hand mesh provided by MRTK, the Snapdragon Spaces SDK also provides a QCHT hand mesh for visualizing virtual hand overlays.

To implement, select the MRTK RightHand Controller object in the scene hierarchy under MRTK XR Rig > Camera Offset > MRTK RightHand Controller.

Drag the hand_right_qcht prefab located in packages/QCHT Unity Interactions/Prefabs/MRTK3/ on the Model Prefab field of the MRTK RightHand Controller

Perform the same process with the left hand.

Hand Tracking Options

NOTE

It is not possible at this time to raycast from the Snapdragon Spaces Controller and both hands at the same time due to a conflict with the OpenXR Controller Profile. As a result, there are two options available.

Option 1: Disable the Left Hand Tracking

To disable the left hand tracking, turn off the Left Hand Controller but leave the Right Hand Controller on in the MRTK XR Rig. If you leave the left hand on, it may still work as a collider, but may not properly work as a raycast interactor. This option allows continued use of the Device Pointer, but the limitation may not be ideal.

Option 2: Disable the Snapdragon Spaces Controller (Two Hand Option)

To disable the Snapdragon Spaces Controller, remove or deactivate the Device Pointer Prefab from the scene if it is present.

Then remove the profile from the OpenXR Settings.

This will flag a warning next to the Base Runtime feature, but can be ignored. This enables both hands to track, but the phone controller will not send any pointer tracking or button pressed messages to the app, as it is a "Hand Tracking only" mode. This is mostly recommended for directly porting over apps from other "hand tracking only" platforms.

MRTK Resources

For more detailed information on MRTK, see the MRTK3 documentation or check out the source at the Github MRTK Project