Image Tracking

This feature is based off AR Foundation's image tracking.

For basic information about image tracking and what AR Foundation's AR Tracked Image Manager component does, please refer to the Unity documentation.

Image targets are fed to the underlying XR plugin through an XR Reference Image Library. The added image has a name which will be useful for identifying a tracked target later on, and an important flag Keep Texture at Runtime which should set to true. This enables the subsystem to pass the texture data to the Snapdragon Spaces Services for tracking purposes.

How to Enable the Feature

In order to use Image Tracking, it has to be enabled in the OpenXR plugin settings located under Project Settings > XR Plug-in Management > OpenXR (> Android Tab) > OpenXR Feature Groups > Snapdragon Spaces > Image Tracking.

Feature Settings

The feature settings can be accessed by clicking on the cog wheel on the right of the feature in the plugin settings.

Image Tracking has 2 feature settings:

• Extended Range Mode: Used by the application to indicate that it prefers tracking to be optimized for long range, possibly at the expense of competing interests.
• Low Power Mode: Used by the application to indicate that it prefers tracking to be optimized for low power consumption, possibly at the expense of competing interests.

Spaces Reference Image Configurator (Script)

The optional Spaces Reference Image Configurator component may be added to the same GameObject as the ARTrackedImageManager. This component provides additional configuration options for each tracked image.

The Tracking Mode field allows one of three different tracking modes to be defined per image:

• Dynamic (Default) - Dynamic mode updates the position of tracked images every frame, and works on moving and static targets. If the tracked image cannot be found, no location or pose is reported. This mode has higher power consumption relative to the other tracking modes. If no Spaces Reference Image Configurator component is found, all tracked images will use this tracking mode by default.
• Static - Static mode is useful for tracking images that are known to be static, which leads to less power consumption and greater performance. These may be images that are affixed to the floor or a wall, for example. This mode is useful for continuing to show augmentations that should still be visible even after the image is no longer visible. The position of images tracked in static mode is never updated, regardless of whether the image has moved or is out of sight.
• Adaptive - Adaptive mode works on static images, but periodically updates the pose of tracked images roughly once every 5 frames if they have moved. If the HMD has moved, the image pose is updated in allginment with the display refresh rate. Tracking for images that are out of sight will eventually be lost. This mode balances power consumption and tracking accuracy for images that will remain fixed in place.

Tracking Mode	Added	Updated	Removed
Dynamic (default)	Frame entered	Every frame	Frame exited
Static	Frame entered	Never	New image tracked
Adaptive	Frame entered	Periodically (roughly once every 5 frames with image movement)	Frame exited

Upon starting the application, each tracked image will initially use the Tracking Mode defined in the Spaces Reference Image Configurator component, but this value can be updated at runtime if necessary, as shown in the simplified example code below.

    public SpacesReferenceImageConfigurator referenceImageConfigurator;
    private Dictionary<TrackableId, ...> _trackedImages = new Dictionary<TrackableId, ...>();
    ...

    string referenceImageName = ...; // referenceImageName to change Tracking Mode for
    TrackableId trackableId = ...; // a TrackableId for an existing tracked instance of this image
    ...

    // first, stop tracking any instances of the tracked image for which the Tracking Mode should be updated.
    // if tracking is not stopped, the instance can (but is not guaranteed to) continue to be tracked using the old Tracking Mode for some time 
    referenceImageConfigurator.StopTrackingImageInstance(referenceImageName, trackableId);
    ...
    
    // then, set the desired tracking mode for all future instances of the tracked image with that referenceImageName
    referenceImageConfigurator.SetTrackingModeForReferenceImage(referenceImageName, SpacesImageTrackingMode.DYNAMIC);
    referenceImageConfigurator.SetTrackingModeForReferenceImage(referenceImageName, SpacesImageTrackingMode.STATIC);
    referenceImageConfigurator.SetTrackingModeForReferenceImage(referenceImageName, SpacesImageTrackingMode.ADAPTIVE);

Swapping Reference Libraries at Runtime

It is possible to change the reference image library used by the AR Tracked Image Manager at runtime without the need of switching scenes. In order to do this, the following must be considered:

• The Image Tracking subsystem needs to be re-started for the library change to take effect. This can be achieved by re-enabling the AR Tracked Image Manager component.
• Tracking modes must be re-synced in the Spaces Reference Image Configurator for the new library by using the component's SyncTrackingModes function. A Dictionary<string, SpacesImageTrackingMode> of reference image names and desired starting tracking modes must be provided as an argument.

danger

Failing to re-apply tracking modes on the Spaces Reference Image Configurator when swapping libraries will cause its SetTrackingModeForReferenceImage function to fail if the reference image name is not found in the previous library.

In such cases, Dynamic mode will be the default tracking mode for the reference image.

The sample code below implements a function SwapTargetLibrary that swaps to a different reference image library at runtime. First the Tracked Image Manager component is disabled, and Image Tracking modes are re-synced in the Spaces Reference Image Configurator component's SyncTrackingModes function. The Dictionary<string, SpacesImageTrackingMode> of reference image names and desired starting tracking modes is provided by the auxiliary CreateTrackingModesDictionary function. Finally, the new reference library is applied to the AR Tracked Image Manager component and the latter is re-enabled.

public ARTrackedImageManager TrackedImageManager;
public SpacesReferenceImageConfigurator ReferenceImageConfigurator;

void SwapTargetLibrary(XRReferenceImageLibrary library)
{
#if !UNITY_EDITOR
    // reference library cannot be updated while the tracked image manager is enabled, generally.
    // but in the editor, when using the Simulation subsystem, disabling the manager causes errors from an AR foundation validator and so should be skipped.
    TrackedImageManager.enabled = false;
#endif
    
    ReferenceImageConfigurator.SyncTrackingModes(CreateTrackingModesDictionary(library));
    TrackedImageManager.referenceLibrary = library;
        
#if !UNITY_EDITOR
    TrackedImageManager.enabled = true;
#endif
}

Dictionary<string, SpacesImageTrackingMode> CreateTrackingModesDictionary(XRReferenceImageLibrary library)
{
    Dictionary<string, SpacesImageTrackingMode> trackingModes = new Dictionary<string, SpacesImageTrackingMode>();
    foreach (var referenceImage in library)
    {
        trackingModes[referenceImage.name] = SpacesImageTrackingMode.DYNAMIC;
    }

    return trackingModes;
}