Image Tracking Sample

This sample demonstrates how to detect and augment image targets found in the real world. For basic information about image tracking and what AR Foundation's AR Tracked Image Manager component does, please refer to the Unity documentation. In order to use this feature it has to be enabled in the OpenXR plugin settings located under Project Settings > XR Plug-in Management > OpenXR (> Android Tab).

How the sample works

First and foremost, make sure to have the Image Tracking feature enabled in the OpenXR project settings.

tip

You can find the used reference images in the Image Targets for Testing section

Image targets are fed to the underlying XR plugin through a XR Reference Image Library. The added image has a name which will be useful for identifying a tracked target later on, and an important flag Keep Texture at Runtime which should set to true. This enables the subsystem to pass the texture data to the Snapdragon Spaces Services for tracking purposes.

Additionally, the image target in the sample's case was measured at 26 cm in height (when printed in DIN A4 or US letter). The correct measures are essential for a correct pose estimation and subsequent placement of an augmentation. Therefore, it must be specified after enabling Specify Size.

In the Import Settings the texture should have the Read/Write flag enabled and the Format set to RGB 24 bit (which may also appear as RGB8). If the Format is set to Automatic, the project's graphics settings may set the format incorrectly.

Adding the ARTrackedImageManager to the ARSessionOrigin GameObject will enable the Image Tracking Subsystem included in the Snapdragon Spaces package. The component provides three fields: a field for the RuntimeReferenceimageLibrary created above, a field for specifying the Max Number Of Moving Images, and a field for defining a prefab to spawn upon detection of a tracked image. The prefab used is a gizmo that indicates the orientation of the tracked image.

By subscribing to the ARTrackedImageManager's method to listen for tracked image changes, the appropriate UI information gets set for the tracking state and position of the tracked image, as seen in the simplified code example below.

danger

For this sample to work the reference image names set in the XR Reference Image Library need to be unique. Any identical names will cause a hash code collision in the _trackedImages Dictionary.

    ...

    private Dictionary<TrackableId, ...> _trackedImages = new Dictionary<TrackableId, ...>();

    public override void OnEnable() {
        ...
        FindObjectOfType<ARTrackedImageManager>().trackedImagesChanged += OnTrackedImagesChanged;
    }

    public override void OnDisable() {
        ...
        FindObjectOfType<ARTrackedImageManager>().trackedImagesChanged -= OnTrackedImagesChanged;
    }

    private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs args) {
        foreach (var trackedImage in args.added) {
            if (trackedImage.referenceImage.name == "Spaces Town") {
                _trackedImages.Add(trackedImage.trackableId, ...);
                ...
            }
        }

        foreach (var trackedImage in args.updated) {
            var info = _trackedImages[trackedImage.trackableId];
            ...
        }

        foreach (var trackedImage in args.removed) {
            _trackedImages.Remove(trackedImage.trackableId);
            ...
        }
    }

The optional Spaces Reference Image Configurator component may also be added to the same GameObject as the ARTrackedImageManager. This component provides additional configuration options for each tracked image.

The Tracking Mode field allows one of three different tracking modes to be defined per image:

• Dynamic (Default) - Dynamic mode updates the position of tracked images every frame, and works on moving and static targets. If the tracked image cannot be found, no location or pose is reported. This mode has higher power consumption relative to the other tracking modes. If no Spaces Reference Image Configurator component is found, all tracked images will use this tracking mode by default.
• Static - Static mode is useful for tracking images that are known to be static, which leads to less power consumption and greater performance. These may be images that are affixed to the floor or a wall, for example. This mode is useful for continuing to show augmentations that should still be visible even after the image is no longer visible. The position of images tracked in static mode is never updated, regardless of whether the image has moved or is out of sight.
• Adaptive - Adaptive mode works on static images, but periodically updates the pose of tracked images roughly once every 5 frames if they have moved. If the HMD has moved, the image pose is updated in allginment with the display refresh rate. Tracking for images that are out of sight will eventually be lost. This mode balances power consumption and tracking accuracy for images that will remain fixed in place.

Tracking Mode	Added	Updated	Removed
Dynamic (default)	Frame entered	Every frame	Frame exited
Static	Frame entered	Never	New image tracked
Adaptive	Frame entered	Periodically (roughly once every 5 frames with image movement)	Frame exited

Upon starting the application, each tracked image will initially use the Tracking Mode defined in the Spaces Reference Image Configurator component, but this value can be updated at runtime if necessary, as shown in the simplified example code below.

    public SpacesReferenceImageConfigurator referenceImageConfigurator;
    private Dictionary<TrackableId, ...> _trackedImages = new Dictionary<TrackableId, ...>();
    ...

    string referenceImageName = ...; // referenceImageName to change Tracking Mode for
    TrackableId trackableId = ...; // a TrackableId for an existing tracked instance of this image
    ...

    // first, stop tracking any instances of the tracked image for which the Tracking Mode should be updated.
    // if tracking is not stopped, the instance can (but is not guaranteed to) continue to be tracked using the old Tracking Mode for some time 
    referenceImageConfigurator.StopTrackingImageInstance(referenceImageName, trackableId);
    ...
    
    // then, set the desired tracking mode for all future instances of the tracked image with that referenceImageName
    referenceImageConfigurator.SetTrackingModeForReferenceImage(referenceImageName, SpacesImageTrackingMode.DYNAMIC);
    referenceImageConfigurator.SetTrackingModeForReferenceImage(referenceImageName, SpacesImageTrackingMode.STATIC);
    referenceImageConfigurator.SetTrackingModeForReferenceImage(referenceImageName, SpacesImageTrackingMode.ADAPTIVE);

Swapping Reference Libraries at Runtime

It is possible to change the reference image library used by the AR Tracked Image Manager at runtime without the need of switching scenes. In order to do this, the following must be considered:

• The Image Tracking subsystem needs to be re-started for the library change to take effect. This can be achieved by re-enabling the AR Tracked Image Manager component.
• Tracking modes must be re-synced in the Spaces Reference Image Configurator for the new library by using the component's SyncTrackingModes function. A Dictionary<string, SpacesImageTrackingMode> of reference image names and desired starting tracking modes must be provided as an argument.

danger

Failing to re-apply tracking modes on the Spaces Reference Image Configurator when swapping libraries will cause its SetTrackingModeForReferenceImage function to fail if the reference image name is not found in the previous library.

In such cases, Dynamic mode will be the default tracking mode for the reference image.

The sample code below implements a function SwapTargetLibrary that swaps to a different reference image library at runtime. First the Tracked Image Manager component is disabled, and Image Tracking modes are re-synced in the Spaces Reference Image Configurator component's SyncTrackingModes function. The Dictionary<string, SpacesImageTrackingMode> of reference image names and desired starting tracking modes is provided by the auxiliary CreateTrackingModesDictionary function. Finally, the new reference library is applied to the AR Tracked Image Manager component and the latter is re-enabled.

public ARTrackedImageManager TrackedImageManager;
public SpacesReferenceImageConfigurator ReferenceImageConfigurator;

void SwapTargetLibrary(XRReferenceImageLibrary library)
{
    TrackedImageManager.enabled = false;
    ReferenceImageConfigurator.SyncTrackingModes(CreateTrackingModesDictionary(library));
    TrackedImageManager.referenceLibrary = library;
    TrackedImageManager.enabled = true;
}

Dictionary<string, SpacesImageTrackingMode> CreateTrackingModesDictionary(XRReferenceImageLibrary library)
{
    Dictionary<string, SpacesImageTrackingMode> trackingModes = new Dictionary<string, SpacesImageTrackingMode>();
    foreach (var referenceImage in library)
    {
        trackingModes[referenceImage.name] = SpacesImageTrackingMode.DYNAMIC;
    }

    return trackingModes;
}