# MRTK3 Unity Setup

This sample scene demonstrates how to integrate MRTK3 into your Unity project. For more information on MRTK, visit the MRTK3 documentation (opens new window).

# Setup a New Unity Project with MRTK3 & Snapdragon Spaces on Windows

In this guide, we will setup and build a brand new MRTK3 Project for Snapdragon Spaces.

Note

  • This tutorial currently only integrates with Snapdragon Spaces Controller Input.
  • Hand Tracking and Gaze Pointer tracking are potentially capable but beyond the scope of this document at this time.
  • Please avoid components that utilize the virtual keyboard and web browser, as those are not supported at this time and could cause erratic behavior.

# Unity Project Setup

Start with the Unity Setup Guide from Prerequisites down to Enable Spaces Features.

TIP

For the Unity version, it is recommended to use Unity 2020.3.35+ LTS.

Once set up, keep your Unity Project open and take note of the filepath location.

# Project Configuration for MRTK

  • Install MRTK requirements for MRTK3:
    • Visual Studio 2019 Community edition or greater (opens new window)
    • Mixed Reality Feature Tool is required to access the MRTK3 packages. This is currently only available on Windows. Download, but do not run yet. (opens new window)
    • You will need .Net 0.5+ Runtime which is included in Visual Studio 16.11.14+

# Import the MRTK3 packages

  • Open the Mixed Reality Feature Tool (MixedRealityFeatureTool.exe)
  • Wait for it to load up, then click the Settings icon in the lower left.
    • Click the Feature tab.
    • Check the box for Show preview releases.
    • Click Ok to confirm.

MRTK Tool

  • Click “Start”.
  • Set the Project Path to the filepath location of the Unity Project.
  • Click “Discover Features”.
    • Click "Select All" for the MRTK3 line item.
    • Click “Get Features”. Some recommended dependencies may be displayed. If so, select those too. Click “Import” when done to download files and update the Unity Project manifest.
    • When finished, optionally close the Feature Tool, or keep it open to make continued adjustments as needed.

MRTK Tool

# Return to Unity to complete package import

  • Switchback to the Unity Project and wait for any importing to occur as a result of the new features being added by the Mixed Reality Feature Tool.
    • If prompted, select I made a backup, go ahead! regarding the XR Interaction Layer Mask.
    • If prompted to restart, choose Yes

MRTK Action Inputs

# Scene Setup

  • Create a new Scene. We will use this to set up the camera and input from scratch.

# Modify the Camera for Snapdragon Spaces

  • Delete the Main Camera from the Scene Hierarchy.
  • Find the MRTK XR Rig Prefab and drag it into the Scene, located at: Packages/com.microsoft.mrtk.input/Assets/Prefabs/MRTK XR Rig.prefab

MRTK Prefab

  • On Main Camera: (MRTK XR Rig -> Camera Offset ->Main Camera)
    • Remove TrackedPoseDriver component
    • Add AR Camera Background component
      • Note that this will automatically add an AR Camera Manager with Auto Focus selected, Light Estimation set to None, and Facing Direction set to World. This is fine.
    • Add AR Pose Driver component

MRTK AR Pose

  • Add an AR Session Object by right-clicking the in the Hierarchy and choosing XR -> AR Session.
    • An AR Session GameObject will be generated and added to the Scene.
      • Keep the default attributes for the AR Session component:
        • Attempt Update: checked
        • Match Frame Rate: checked
        • Tracking Mode: Position and Rotation
      • An AR Input Manager component will also be included on the GameObject. If it is not, add it.

MRTK Main Camera

# Remove MRTK controllers

  • Under Camera Offset: (MRTK XR Rig -> Camera Offset)
    • Turn off the MRTK RightHand Controller
    • Turn off the MRTK LeftHand Controller
    • Turn off the MRTK Gaze Controller

MRTK hierarchy

# Configure for Snapdragon Spaces Controller Input

As stated above, this guide is only for using the Companion Controller as device input. To set up for Companion Controller, follow the steps below.

  • On MRTK XR Rig (top-level GameObject):
    • In the Input Action Manager Component, change the Action Assets dropdown's Element 0 to: Input Actions located at: Samples/Snapdragon Spaces/[version ID]/Core Samples/Shared Assets/Input Actions. This will map the input actions to the ones set up from the Snapdragon Spaces Core Samples, allowing for proper connection to the Companion Controller.

MRTK Input Actions

  • Now add the Device Pointer Prefab located at: Samples/Snapdragon Spaces/[version ID]/Core Samples/Shared Assets/Prefabs/Interaction/
    • In the XR Interactor Line Visual Component, change the Line Width to 0.005.

MRTK hierarchy

# Finally, set the MRTK Profile.

  • Create a copy of the MRTKProfile found in Packages->MRTK Core Definitions->Configuration->Default Profiles and drop it into the projects Assets directory.

MRTK Profile

  • Open (Edit -> Project Settings -> MRTK3)
  • Drag the copy into the Profile section under the Android tab.
  • Uncheck the MRTK Hands Aggregator Subsystem
  • Uncheck the Subsystem for Hand Synthesis

MRTK Profile

# Optional: Add Interactive Components to test out MRTK functionality

# Selectable Object

  • Add a 3D Cube to the scene.
    • Set the Position to (0, 0, 1.5)
    • Rotation to (45, 45, 45)
    • Scale to (0.5, 0.5, 0.5)
  • Add Component "ObjectManipulator" to allow grabbing from the laser pointer.

# MRTK Button Group

  • Test out the MRTK Prefab Interaction components by adding a ButtonGroup_32x32mm_H3 prefab into the Scene.
  • This may prompt you to install TextMeshPro. If so, select Import TMP Essentials.
  • Select the ButtonGroup in the Hierarchy and in the Inspector, set the Transform Position to 0,0,0.5. Press Play to process scripts and visualize the button faces, then Stop to return to Editing.
  • Expand the ButtonGroup to find the three PressableButton_32x32mm_IconAndText objects and expand them one at a time.

MRTK Profile

  • For the first one, select the top-level button object, find the Pressable Button component in the Inspector, click + for OnClicked(), set it to the Cube in the Scene, and set the action to GameObject.SetActive with the checkbox unchecked.
    • Under the child CompressableButtonVisuals:
      • Find the UX.Button.Icon.Char child object and change the SpriteRenderer`s Sprite to an empty circle.
  • For the second one, select the top-level button object, find the Pressable Button component in the Inspector, click + for OnClicked(), set it to the Cube in the Scene, and set the action to GameObject.SetActive with the checkbox checked.
    • Under the child CompressableButtonVisuals:
      • Find the UX.Button.Icon.Charchild object and change the SpriteRenderer`s Sprite to a filled circle.
      • Find the TextMeshPro child and set the Text Input to “Show”.
  • For the third button, you will first need to create a new Script and add it to the Cube. The script should have a publicly accessible function such as
    ...
    public void Quit() {
        Application.Quit();
    }
    ...
  • Still on the third button:
    • Select the top-level button object, find the Pressable Button component in the Inspector, click + for OnClicked(), set it to the Cube in the Scene, and set the action to Quit() from the new script.
      • Under the child CompressableButtonVisuals:
        • Find the UX.Button.Icon.Char child object and change the SpriteRenderer`s Sprite to a Quite button.
        • Find the TextMeshPro child and set the Text Input to “Show”.

MRTK Profile

# Testing to device

  • Configure the Build Settings and Player Settings to build the app.
  • Build the app from the Build Settings window and save your app locally.
  • Connect your Android Device and approve USB Debugging if prompted.
  • Install the apk to your device.
  • Long-press the App Icon, tap "App Info", and approve all of the Permissions.
  • Plug in the glasses, and once fully loaded, run the app.

# MRTK Samples

For additional insight into the available MRTK3 user interface components, it is recommended to download the MRTK3 Samples and configure them for Snapdragon Spaces. Follow the steps below to do this.

# Get the MRTK3 Samples

  • Clone the repo or download the mrtk3 branch at: https://github.com/microsoft/MixedRealityToolkit-Unity/tree/mrtk3
  • Extract or Clone this repo in a location with a short filename. This repo has many folders with long names, which can result in file names becoming too long and causing issues with Unity.
  • Do not move any of the files in this repo, as there are many dependencies and changing a file’s position will cause issues. If you need to move the repo, move the entire repo altogether.

# Open Project and integrate the Snapdragon Spaces:

  • Add “MRTKDevTemplate” from the MRTK3 Samples “UnityProjects” folder in the Unity Hub.
  • Open the Unity Project.
  • Import the Snapdragon Spaces package as in the Unity Setup Guide.
  • Select a Scene to open. Recommended Scene is HandInteractionExamples.
  • Configure the Scene to modify the camera and input as above.
  • Build and deploy to device.