Chapter 1 – Design Pillars for creating Augmented Reality and/or Mixed Reality Apps

Hello guys!

As you know i am creating my own personal project to build a Mixed Reality App. I have the concept, the requirements and all necessary items to develop, but… where do i start?


“Understand the user’s environment”.

If you are in the same situation than me, one of the things i suggest you to focus on, is make sure you understand that AR/MR exists OUTSIDE of our device. So your design work should exist outside of the device as well, so put away your 2D UI ideas, its not easy but once you get the concept you kind of feel free to create amazing things.

So we have to focus, on creating a sketch of the actual environment the user will be in, so you should be sketching living rooms, and tables, and outdoor spaces.

Then as you sketch the users environment, you start to sketch the various objects that are going to be interacting with in that environment. You can sort of think of AR / MR as having a lot of the same challenges as responsive design for the web in terms of different window sizes, but it’s even more complicated because now you have responsive design for 3d spaces that are the user’s actual living room.

So you want to sketch the user for scale, to get a sense of how you are going to start crafting this experience. The user could be very large relative to the AR objects or very small and then you wanna think how that user is going to move around in that environment.

WhatsApp Image 2020-01-19 at 18.53.59

“Plan for the user’s environment”

So now that we understand how to design for the environment let’s think about how to design for user movement. Its important to set users expectations ahead of time about the physical scale of the experience. Lets dive in into the three of them.

IMPORTANT TIP “It’s ok to design beyond the bounds of the screen.”

Users are more familiar with 2D mobile applications that do not require the user to move as a form of interaction, it can be very challenging getting users to realize that they can move around. So this is one important point, to create movement UI elements at the beginning can help users understand that he can move around those objects. And when objects move outside the screen, its replaced with a marker that indicates there is an object on that place. I found this concept diagram which i thought was pretty interesting and the most accurate. We do have a PRIMARY FOV but we also have the possibility to move our user around to other views. It helps to have this in mind when designing.

AR Kitchen Perspective

Another important thing to consider when designing you APP is determining how much space the user needs. Right now designers indicate that experiences fall into three different sizes.


Table Scale

In this case, the experience is able to scale to the smallest of surface so that many users are able to interact/enjoy with the experience.

Room Scale

In this case, it expands the impact of AR, so that content will start to feel life sized and you are able to do a lot more with the space that is available.

World Scale

In this case, it has no limits it allows users to appreciate AR in whatever area they see fit, and this is an area where they particularly are excited about because for procedurally generated content in world scale.

So, as conclusion, how much you require users to move around the space will depend on the type of app you are creating.

Onboard users by initializing smoothly

Another thing you want to think about is deciding if users are able to easily move the objects after they’ve been placed, or if they are more permanent objects. The answer of course will depend on the app you are developing (Augmented Reality or Mixed Reality).

For permanent object we have to consider using anchors for these persistent object placements. And you still wanna enable the user to move the object perhaps trough a menu or some type of re-anchoring flow. So set expectations ahead of time that the objects will be sort of stuck in one place for a while as you interact with the object.


Design natural object interactions

One other thing we have to consider, is thinking about hot to solve problems for user’s behavior, even when its unintentional. So one of the things recommended is giving users feedback on object collisions.

This is a problem that I faced while trying to build prototypes, when i was getting more closer to an object it disappeared. Why did this happen? Basically if I got really close it mean i was inside the object, in this case its recommended that when this happens, you change the users view with a sort of effect, to let the user understand that its inside the object.

It is also important to give feedback to the users, this will be helpful when we think of each stage of the user journey, even as it relates to surface feedback.

Lets think how an object might get on the scene.


Balancing on-screen and volumetric interface design

The last thing you want to remember, is that the device is the user’s viewport. They are actually using the device to look out into the scene and see the application. And because of that you don’t actually want to place a lot of the 2d UI on the screen that’s actually going to obscure the user’ s view onto your application.

Only use screen surface UI for

1 – Controls with a high frequency of use

2 – Controls that require fast access

Hope this helps you start thinking outside the box when designing your next app.


One thought on “Chapter 1 – Design Pillars for creating Augmented Reality and/or Mixed Reality Apps

Add yours

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at

Up ↑

Create your website with
Get started
%d bloggers like this: