Dual Render Fusion Components

This section details the functionality of all new components included for Dual Render Fusion.

The Apply Project Validator Fixes step from the Dual Render Fusion Setup Guide adds additional components to the scene.

Additional helpers are available to create these game objects manually. These helpers can be found by right-clicking in the scene hierarchy and navigating to XR > Dual Render Fusion. Or through the menu bar GameObject > XR > Dual Render Fusion.

Dynamic OpenXR Loader component

The Dynamic OpenXR Loader component allows the application to load OpenXR after glasses are connected, similar to the concept of how headphones work. This component works alongside the Spaces Glass Status component which detects the presence of supported glasses and then starts up OpenXR if configured to do so.

Several callbacks are available to control the behavior of the user interface when OpenXR begins connecting. These callbacks are exposed to developers through the Fusion Lifecycle Events component.

SCENE TRANSITIONS

The Dynamic OpenXR Loader is designed to survive scene transitions.

It should exist on a root-level GameObject which will be set to DontDestroyOnLoad automatically at runtime.

The Dynamic OpenXR Loader is intended to be created early in the lifecycle of an application, and to persist until the application ends, even if there is no XR content to display in a scene when the glasses connection is detected.

DISABLE INITIALIZE XR ON STARTUP

To use the Dynamic OpenXR Loader component, Initialize XR on Startup must be disabled in Project Settings > XR Plug-In Management > Initialize XR on Startup. This is usually configured with the automatic setup using the Configure Fusion project menu item.

UPGRADING AN EXISTING DUAL RENDER FUSION PROJECT

If upgrading an existing project, older versions of this script should be replaced by the new version included in the Snapdragon Spaces SDK package for Unity version 0.23.0 or later.

See the Dual Render Fusion Migration Guide for more information.

Dynamic OpenXR Loader Properties

• Auto Start XR On Display Connected: When enabled, the application will automatically launch OpenXR when any supported glasses are connected. If this is disabled, the developer must call the StartOpenXR method on the Dynamic OpenXR Loader component when XR is needed.

  Developers can manually start and stop OpenXR by calling:

  DynamicOpenXRLoader.Instance.StartOpenXR();
  ...
  DynamicOpenXRLoader.Instance.StopOpenXR();

  StartOpenXR will only succeed if any supported glasses are connected. See Spaces Glass Status component.

  WARNING

  When OpenXR begins starting, the main thread is blocked for a short period of time. It is recommended to display UI elements signaling that OpenXR is starting, and for these elements to show a confirmation state upon completion of loading.

• Auto Manage XR Camera: When enabled, the game objects containing the AR Session and XR Origin are enabled and disabled as needed when OpenXR starts or stops.

  tip

  When using AR Foundation 5.0 or later the Dynamic OpenXR Loader component will instead find an XROrigin component to enable and disable.

• Cannot Display Overlay Dialog Options Provider: This optional field can be used to customize the text shown to the user when the Display over other apps permission is not allowed for the OpenXR Runtime application. When this permission is not allowed the Dual Render Fusion content cannot be drawn.

• Runtime Camera Permissions Dialog Options Provider: This optional field can be used to customize the text shown to the user when the camera permission is not allowed for the OpenXR Runtime application. When this permission is not allowed some features won't work. This is the list of the required permissions per feature.

  This field accepts a Simple Dialog Options Provider component. This component should be implemented by the application developer and must implement the method:

  public override SimpleDialogOptions GetDialogOptions()

  The SimpleDialogOptions return type contains strings to customize each text entry:

  • Title
  • Message
  • PositiveButtonText
  • NegativeButtonText

  This text can be localized by implementing a dialog options provider which works with the desired localization toolchain. e.g. if using the Unity.Localization package, the developer implementation of the provider can simply look up the localized text in the string table.

  public class LocalizedSimpledDialogOptionsProvider : SimpleDialogOptionsProvider
  {
      public LocalizedString Title;
      public LocalizedString Message;
      public LocalizedString PositiveButton;
      public LocalizedString NegativeButton;
  
      public override SimpleDialogOptions GetDialogOptions()
      {
          return new SimpleDialogOptions()
          {
              Title = this.Title.GetLocalizedString(),
              Message = this.Message.GetLocalizedString(),
              PositiveButtonText = this.PositiveButton.GetLocalizedString(),
              NegativeButtonText = this.NegativeButton.GetLocalizedString()
          };
      }
  }

  It is recommended to attach an implemented Simple Dialog Options Provider component directly to the Dynamic OpenXR Loader component in the scene.

  

  If no dialog options provider is given, the following default (English) text will be used for Cannot Display Overlay Dialog Options Provider:

  • Title - OpenXR Runtime permissions
  • Message - The 'Display over other apps' permission is required for the OpenXR runtime to render XR content. Select 'Configure' to grant this permission manually by selecting 'Snapdragon Spaces Services' from the list shown.
  • Positive Button - Configure
  • Negative Button - Not Now

  If no dialog options provider is given, the following default (English) text will be used for Runtime Camera Permissions Dialog Options Provider:

  • Title - OpenXR Runtime permissions
  • Message - Camera permissions for the OpenXR runtime are not granted. Some features might not work. Would you like to grant the camera permissions now? If you select yes, this application will close and you will need to re-start it.
  • Positive Button - Yes
  • Negative Button - Not Now

Spaces Glass Status component

The Spaces Glass Status component is used for accessing the information of the device type the application is running on and to get the information related to the connection status of the glasses.

RECOMMENDATION

While the Spaces Glass Status component can exist on its own, it is recommended to work with the Dynamic OpenXR Loader component to make managing the lifecycle of XR content simpler.

SCENE TRANSITIONS

The Spaces Glass Status is designed to survive scene transitions.

It should exist on a root-level GameObject which will be set to DontDestroyOnLoad automatically at runtime. (This is likely to be the same object as the Dynamic OpenXR Loader).

The Spaces Glass Status is intended to be created early in the lifecycle of an application, and to persist until the application ends, even if there is no XR content to display in a scene when the glasses connection is detected.

This component exposes callbacks to developers through the Fusion Lifecycle Events component.

Developers can additionally query this data through code at runtime as shown below.

Spaces Glass Status Available Data

Glass Connection State:

var connectionState = SpacesGlassStatus.Instance.GlassConnectionState;

Retrieve the connection state of the glasses.

• The glass connection state is either Disconnected or Connected.

Glass Active State:

var activeState = SpacesGlassStatus.Instance.GlassActiveState;

Retrieve the active state of the glasses.

• The glass active state is either Active or Idle.
• Glasses cannot be Active if they are Disconnected.
• Idle indicates that the glasses are not being worn - this can be triggered due to e.g. proximity sensor timeout. This can be used as a hint to save power by deprioritising rendering by reducing quality, or reducing workload on high-intensity code sections.

Device Type:

var deviceType = SpacesGlassStatus.Instance.DeviceType;

Retrieve the device type of the glasses.

• Device type is broken up into broad categories of device. These categories can help inform the application about how best to display content for a given category.
• Device Type can be None, Aio, Wired, or Wireless.
  • None indicates that the glassConnectionState is currently Disconnected.
  • Aio is an All-In-One glass type (such as an MR or VR headset viewer).
  • Wired is any AR glass type connected via a cable to a smartphone.
  • Wireless is any AR glass type connected to a smartphone wirelessly.

Spaces Host View component

The Spaces Host View component dynamically disables the smartphone display when the application is run on MR/VR devices which don't support Dual Render Fusion.

It additionally provides callbacks the developer can use to enable/disable additional functionality as needed, such as enabling an alternative UI for MR/VR devices. These callbacks are exposed to developers through the Fusion Lifecycle Events component.

This component includes the camera which will render to the smartphone display.

SCENE TRANSITIONS

The Spaces Host View is designed to survive scene transitions.

It should exist on a root-level GameObject which will be set to DontDestroyOnLoad automatically at runtime.

The Spaces Host View is intended to be created as early in the lifecycle of an application as possible, and to persist until the application ends. If an application created to use Dual Render Fusion is launched on an incompatible device, such as an MR/VR headset, the Spaces Host View will broadcast events early. These should be handled to e.g. hide smartphone specific displays.

UPGRADING AN EXISTING DUAL RENDER FUSION PROJECT

If upgrading an existing project, this component should be added to the GameObject containing the camera which will render to the smartphone.

See the Dual Render Fusion Migration Guide for more information.

USING UNIVERSAL RENDER PIPELINE

If using Universal Render Pipeline, the Target Eye in the camera component must be set to None.

Fusion Lifecycle Events component

The Fusion Lifecycle Events component gathers all the events related to the Dynamic OpenXR Loader, Spaces Glass Status, and Spaces Host View components.

SCENE TRANSITIONS

The Fusion Lifecycle Events is not designed to survive scene transitions, unlike the classes above. It should be instantiated per scene. In each scene, the developer can configure how each of the exposed events should behave.

tip

The developer has complete control over how each scene behaves when external glasses are connected or disconnected.

• In a menu scene there might be no additional XR content to display - but there may be UI elements which show that the glasses have been successfully detected.
• In scenes which require XR content glass disconnection might require aborting back to a menu scene, or displaying a popup on the smartphone to indicate that the application cannot continue to run unless the glasses are reconnected.

Fusion Lifecycle Events Properties

When Rebroadcast On Scene Load is enabled, the Fusion Lifecycle Events component can optionally rebroadcast the events On OpenXR Available, On OpenXR Unavailable, On OpenXR Started and On OpenXR Stopped when switching scenes. This allows a scene an opportunity to configure correctly according to the current capabilities of the connected devices when the scene has finished loading.

Events

On OpenXR Available

• This event is broadcast when the Dynamic OpenXR Loader component is enabled, as well as before attempting to start OpenXR, and after stopping OpenXR.
• On OpenXR Available will be broadcast at these times when the glasses are connected and fully detected. This indicates that OpenXR can be initialized, but OpenXR is not yet actively running.
• This event occurs upon detected connection of the glasses with a compatible runtime available, however note that OpenXR still needs to be initialized before the application can access OpenXR features.
• Glass connection status can be queried at any time through SpacesGlassStatus.Instance.GlassConnectionState.

On OpenXR Unavailable

• This event is broadcast when the Dynamic OpenXR Loader component is enabled, as well as before attempting to start OpenXR, and after stopping OpenXR.
• On OpenXR Unavailable will be broadcast at these times when the glasses are disconnected. This indicates that OpenXR cannot be initialized, and is not actively running.
• No OpenXR path is detected, indicating that the glasses are no longer connected and should be reconnected in order to enable OpenXR features.
• Glass connection status can be queried at any time through SpacesGlassStatus.Instance.GlassConnectionState.

On OpenXR Starting

• This event is broadcast after OpenXR has been initialized, just before OpenXR subsystems are started.
• At this time it should be clear which OpenXR features are enabled, and have been initialized correctly, but subsystems will not be running.

On OpenXR Started

• This event is broadcast after OpenXR subsystems have started.
• All XR content should now be usable.
• This is the event developers should subscribe to when wanting to start up optional XR content in the scene.
• After this event is broadcast, OpenXR features can be safely used until a Stopping event is broadcast.

On OpenXR Stopping

• This event is broadcast before OpenXR subsystems are stopped and before the OpenXR loader is de-initialized.
• Developers should subscribe to this event in order to shut down content which requires OpenXR while it is safe to do so.
• When this event is called the developer must immediately stop using any OpenXR features.

On OpenXR Stopped

• This event is broadcast after OpenXR subsystems are stopped, and the OpenXR loader has been de-initialized.
• When this event is received, OpenXR is not loaded.
• Features can be queried to see if they are intended to be used, but are not actually usable.
• OpenXR subsystems are not running.
• Immediately after this event, availability will be broadcast indicating whether OpenXR can be restarted (are glasses connected or not).

On Idle

• This event is broadcast when glasses are idle, as in they appear to have been set down and are not being worn (e.g. this is triggered due to proximity sensor timeout).
• This can be used as a hint to save power by reducing rendering quality, or reducing priority of updates to xr only content.
• Glass active/idle status can be queried at any time through SpacesGlassStatus.Instance.GlassActiveState.

On Active

• This event is broadcast when glasses are active, as in they are being worn and displaying content.
• Glasses cannot be active if they are disconnected.
• Glass active/idle status can be queried at any time through SpacesGlassStatus.Instance.GlassActiveState.

On Host View Enabled

• The On Host View Enabled event is fired when using a device which supports Dual Render Fusion. The application can then choose to display or hide elements as appropriate. E.g. to enable the Dual Render Fusion host view on AR devices.
• This is not an indication about whether a supported AR device is connected to the host device.

On Host View Disabled

• The On Host View Disabled event is fired when using a device which does not support Dual Render Fusion. The application can then choose to display or hide elements as appropriate. E.g. to enable alternative user interface elements on MR/VR devices.
• This indicates that there is no valid host device (smartphone) and that SpacesGlassStatus.Instance.DeviceType is likely Aio.

Fusion Simulator component

For enabling Dual Render Fusion Editor simulation, a Fusion Simulator component must be added to the scene. This component, together with the Spaces Host View, handles the camera logic for correctly displaying Dual Render Fusion applications in the Unity Editor on Display 1 and 2.

Fusion Screen Setup component

Fusion Screen Setup contains the logic for forcing the screen orientation to portrait or landscape on the smartphone display.

This is an optional component.

It demonstrates a correct way to make calls to set the smartphone screen orientation in Unity at runtime through calls such as Screen.orientation = ScreenOrientation.Portrait;. Calls to set Screen.orientation must ensure that the Dual Render Fusion feature is enabled, or they may change the orientation of MR/VR headset screens which will behave badly.

Spaces Permission Helper component

The Spaces Permission Helper component can be used for checking that the OpenXR Runtime application is installed, and that the Unity application has the required camera permissions. See Handling App Permissions With Dual Render Fusion for more information about this topic.

Adding this component to a Game Object in the scene will:

• Check if the OpenXR Runtime application is installed. If it is not installed, the event On Runtime Not Installed is triggered.
• Check if the Unity application has been granted camera permissions if a feature that requires it is enabled. If permission was not granted, the event On Application Camera Permissions Not Granted is triggered.
• Check when the Unity application has been granted camera permissions. Once camera permissions have been granted, the event On Application Camera Permissions Granted is triggered.

Developers can react to these events to determine how the application should behave. This component has a public function which can trigger the application camera permissions request called RequestApplicationCameraPermissions.

tip

The OpenXR Loading Test sample scene contains a sample usage of this component.

Components From Samples

Some components are only included in the Samples.

Companion Controller

Some developers will want to migrate existing Snapdragon Spaces projects to use Dual Render Fusion and take advantage of the Companion Controller capabilities of gyroscopic 3DOF pointing. Follow these steps to add this to a project.

To replicate the Companion Controller in Unity, several Assets are needed as well as a few scripts.

1. Import the Samples.

2. If needed, import TextMeshPro whenever prompted.

3. From Assets > Samples > Snapdragon Spaces > [Snapdragon Spaces version] > Fusion Samples > Prefabs, drag the CanvasControllerCompanion Prefab from the imported samples into the Hierarchy.

4. Two game objects may need to be hooked up to the XR Camera since that is not part of the prefab. If not set, the XR Origin will be automatically detected at runtime to use the attached Camera. Alternatively, they can be set manually in the ControllerPositionHeadMirror, under the Ar Camera field.

   <img alt="Controller Position Head Mirror" src={require("./images/fusion/companion_head_mirror.png").default} style={{maxHeight:700}}/>

   • In the GyroRotationReceiver, under the Ar Camera field.

     

5. Ensure that Active Input Handling is set to Both to enable the gyroscope controls:

   

   This should be handled correctly by the Apply Project Validator Fixes step from the Dual Render Fusion Setup Guide.

6. From Assets/Samples/Snapdragon Spaces/[Snapdragon Spaces version]/Fusion Samples/Controller/Input Actions add ControllerPrefabInputActions as an additional Action Asset to the Input Action Manager.

   

   If there is no Input Action Manager in the Hierarchy, create an empty GameObject and add the Component, first.

7. To utilize Menu Button functionality, it is possible to just use the standard Unity Input System and add an On Click() event to the Button object in the Hierarchy representing the Menu Button.

   

8. To add more options in the Settings Menu, temporarily turn on the Settings Overlay game object and add menu items to the Scroll Box list. (Currently, the About button does not have functionality and is merely there as an example.)