AR, MR, VR, and XR: Working With the Virtuality Continuum

NAB

If you’re a thinking of a future career, you could do worse than training your focus on how we are all going to interface with the internet as physical and digital worlds increasingly blend.

article here

User experience (UX) design for virtual reality, augmented reality and mixed reality is going to be essential for the success of our navigation in and enjoyment of the blurred reality-meets-metaverse. The fundamentals, according to UX experts, is to clearly distinguish AR, MR and VR from any other XR technology, which exist on a spectrum of immersiveness.

“For all these technologies to work, they all need to feel real,” explains Laia Tremosa, senior content strategist at Interaction Design Foundation. “The difference between them is how much they rely on physical or digital elements.”

Much of what Tremosa outlines will be common knowledge to anyone who has worked in or explored the rise of VR and AR, but being basic doesn’t negate its importance. It underlines the Foundation’s message that successful UX starts from a rock-solid understanding of reality.

Here, for example, are footballers from Fulham FC attempting to walk a virtual plank:

Typically, VR takes advantage of the visual and auditory systems. However, there is an even greater sense of presence and immersion if you add other senses.

The Foundation revives the virtuality continuum, a concept introduced by researchers Paul Milgram and Fumio Kishino in 1994. The continuum contains the full spectrum of possibilities between the entirely physical world or real environment and the fully digital world or virtual environment. In a continuum, adjacent parts are almost indistinguishable, but the extremes are very different.

“This theoretical framework can help you visualize and understand the differences between the various technologies that exist today and those that are yet to be invented,” says Tremosa.

“For example, you can simplify the concept and think of a spectrum of immersion, where one end is low immersion and the other end is high immersion. Then, you can situate the technologies according to the degree of immersion they provide.”

Sometimes the exact limits of the various technologies are not a hundred percent clear, and they may overlap.

“However, the critical part is that you understand the region they occupy in the spectrum.”

So, using the continuum as our guide, extended reality (XR) should be understood as encompassing any technology that alters reality by adding digital elements to the physical or real-world environment by any extent, blurring the line between the physical and the digital world.

XR includes AR, MR, VR, and any other technology — “even those that have yet to be developed — situated at any point of the virtuality continuum.”

Given that XR technologies keep evolving, and their full potential remains to be seen, for the UX designer specializing in XR experiences the difficulty lies in the lack of defined standards.

AR, in this schema, is described as a technology that allows the superposition of digital elements into the real-world environment. In the AR experience, you can see a composite view of physical or real-world elements and digital elements. However, there is no interaction between the digital elements and the physical world elements.

Mixed reality (MR) allows not only the superposition of digital elements into the real-world environment but also their interaction. In the MR experience, the user can see and interact with both the digital elements and the physical ones. Therefore, MR experiences get input from the environment and will change according to it.

“In MR experiences, the user can interact with both digital and physical elements. MR differs from AR — where digital and physical elements don’t interact — and VR — where the physical or real world is completely blocked out.”

Many people struggle with the fact that VR experiences generate true emotional responses even if we know it’s “fake.”

Going back to the video of those footballers walking the plank — you can see that the UX designer has added a physical plank on the ground. According to Tremosa, this activates the sense of touch, meaning that fewer people will be able to accomplish the task — even though they are aware that the heights are not real.

“As all these technologies keep evolving and new ones emerge, UX designers need to keep learning and understanding the characteristics of each one to deliver the best possible user experience.”