Locate Holograms with Spatial Anchors and Mixed Reality Toolkit in IOS

Hi everyone! As promised below you will find a simple tutorial on how to use Spatial Anchors in an iOS app with Mixed Reality Toolkit.

What is Spatial Anchors for?

The way in which it is possible to share a hologram with other devices (we saw in previous tutorials that we are specifically IOS, Android and Hololens 2), is through the use of Spatial Anchors.

Spatial Anchors can be located by multiple devices in the same place and at the same time. With this application, if two different devices connect to the same session, they can locate the saved Anchors.

Let’s start!

This video shows the result of this tutorial.

Software requirements

• Unity version 2018.4.14f1
• Microsoft.MixedReality.Toolkit.Unity.Foundation.2.2.0.unitypackage
• Microsoft.MixedReality.Toolkit.Unity.Providers.UnityAR.2.2.0.unitypackage
• Azure Spatial Anchors 2.1.1
• MRTK HoloLens 2 Unity Tutorials Assets Azure Spatial Anchors v2.1.0.1

Aditional Pachages for IOS

• Ar Foundation version 1.5.0 preview 6
• ARKit XR Plugin v.2.1.2

Creating the App

1. Create a new project and import all packages listed before. If you added only the items mentioned before, your Panel should look like this.

2. Great. Now lets prepare everything in our scene. Select Mixed Reality Toolkit in your menu, then Add scene and configure. once scene was added select it and in the Inspector panel, make sure you have DefaultHololens2ConfigurationProfile

3. In our Panel, lets go to Asset > MRTK.Tutorials.AzureSpatialAnchors > Prefabs and select: ButtonParent.prefab, DebugWindow.prefab and ParentAnchor.prefab

4. Drag the three items to your scene

5. Open the Button Parent Object, and select the “Start button asset” button. Once selected go to the inspector panel and search under Events > OnPress. Then press the (+) button to add another action to the event.

6. With the StartAzureSession button selected, drag the “Parent Anchor” Object to the “None(Object)” under the Runtime Only new row.

7. Then select the “No function” and go to “Anchor Module Script > Start Azure Function ()”

8. With the same button (point 6) selected, scroll down in the Inspector panel until you find the events area and repeat the drag and drop until you see something like this.

9. Ok, we need to repeat steps from 6 to 7 for the other buttons indicating:

• • StopAzureSession –StopAzureSession() function.
  • CreateAnchor – CreateAzureAnchor() function, then drag the ParentAnchor again into the empty None (Game Object) field.
  • Start Looking for Anchor – FindAzureAnchor() function.
  • Delete Azure Anchor – DeleteAzureAnchor() function.
  • Delete Local Anchor  – RemoveLocalAnchor() function, then drag the ParentAnchor again into the empty None (Game Object) field.

10. Finally select the Parent Anchor and in the inspector panel copy the keys from your Spatial Anchors Azure Service.

We now have our app ready lets publish it.

Fix Bugs – Spatial Anchors with MRTK to IOS.

1. Make sure you have your scene selected to publish and platform IOS.

2. Make sure that Strip engine code is UNCHECKED.

 

3. Select “Mixed Reality Toolkit” in your scene, go to Inspector and Clone configuration.

4. Then, in Camera clone the Camera System Type

5. Once you clone it, press the “Ass camera settings provider” which will add a New camera settings, press the small (-) and click Type.

6. Under Type > Microsoft Mixed Reality Toolkit Experimental Unity AR > Unity AR Camera Settings

If you try to publish your app right now you will get the following error.

————————————————————————————————————–

Need an enabled ARReferencePointManager in the scene

SpatialAnchorManager is improperly configured. It has been disabled.

EntryPointNotFoundException: ssc_anchor_locate_criteria_create

NullReferenceException: Object reference not set to an instance of an object.

XR: OpenVR Error! OpenVR failed initialization with error code VRInitError_Init_PathRegistryNotFound: “Installation path could not be located (110)”!

Need an enabled ARReferencePointManager in the scene
UnityEngine.Debug:LogError(Object)
Microsoft.Azure.SpatialAnchors.Unity.<IsValidateConfiguration>d__23:MoveNext() (at Assets/AzureSpatialAnchors.SDK/Scripts/SpatialAnchorManager.cs:383)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start(<IsValidateConfiguration>d__23&)
Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:IsValidateConfiguration()
Microsoft.Azure.SpatialAnchors.Unity.<EnsureValidConfiguration>d__17:MoveNext() (at Assets/AzureSpatialAnchors.SDK/Scripts/SpatialAnchorManager.cs:125)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start(<EnsureValidConfiguration>d__17&)
Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:EnsureValidConfiguration(Boolean, Boolean)
Microsoft.Azure.SpatialAnchors.Unity.<Start>d__39:MoveNext() (at Assets/AzureSpatialAnchors.SDK/Scripts/SpatialAnchorManager.cs:654)
System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start(<Start>d__39&)
Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:Start()

SpatialAnchorManager is improperly configured. It has been disabled.
UnityEngine.Debug:LogError(Object)
Microsoft.Azure.SpatialAnchors.Unity.<EnsureValidConfiguration>d__17:MoveNext() (at Assets/AzureSpatialAnchors.SDK/Scripts/SpatialAnchorManager.cs:133)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start(<EnsureValidConfiguration>d__17&)
Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:EnsureValidConfiguration(Boolean, Boolean)
Microsoft.Azure.SpatialAnchors.Unity.<Start>d__39:MoveNext() (at Assets/AzureSpatialAnchors.SDK/Scripts/SpatialAnchorManager.cs:654)
System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start(<Start>d__39&)
Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:Start()

EntryPointNotFoundException: ssc_anchor_locate_criteria_create
Microsoft.Azure.SpatialAnchors.AnchorLocateCriteria..ctor () (at Assets/AzureSpatialAnchors.SDK/Plugins/Common/AzureSpatialAnchorsBridge.cs:1487)
AnchorModuleScript.Start () (at Assets/MRTK.Tutorials.AzureSpatialAnchors/Scripts/AnchorModuleScript.cs:41)

NullReferenceException: Object reference not set to an instance of an object
Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager+<Start>d__39.MoveNext () (at Assets/AzureSpatialAnchors.SDK/Scripts/SpatialAnchorManager.cs:657)
— End of stack trace from previous location where exception was thrown —
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <e1319b7195c343e79b385cd3aa43f5dc>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__6_0 (System.Object state) (at <e1319b7195c343e79b385cd3aa43f5dc>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at C:/buildslave/unity/build/Runtime/Export/UnitySynchronizationContext.cs:155)
UnityEngine.UnitySynchronizationContext:ExecuteTasks() (at C:/buildslave/unity/build/Runtime/Export/UnitySynchronizationContext.cs:105)

————————————————————————————————————–

SOLUTION

The solution for this is the following.

1. Select Mixed Reality Toolkit in your scene.

2. Under Inspector Panel, select “Add component”

3. Search for “AR Reference Point Manager” and add it. Apparently this is a bug not fix yet.

Ok then now lets try to publish it.

Publish Spatial Anchors App to IOS

1. Go to File > Build and Settings > Build.

2. Once you have your build done in the folder selected, open the Terminal. Go to the folder in which you have your xcode project

write :   pod install  –repo-update

then: open ./Unity-iPhone.xcworkspace

3. This will open your XCode workspace and you can then publish to your device.

How does this work?

When you start the app you can move the object with you selected to be the Parent anchor in our tutorial is the orange cube. Once we move it, then we press the button Create anchor which will require you to scan your environment until it reaches 100%. Once it created the anchor, you will see a message indicating the id of the anchor, you can now disconnect from the session.

Move the cube to another place, then connect to your session again and press… Start looking for anchors, once the service located one, your Parent Anchor will move from the new place where you put it to the anchor that was identified.