IEC E-tech
article here
Artificial intelligence (AI) is enabling extended reality
environments (XR) to push boundaries even further. While standards for XR
exist, more will be required to foster interoperability between both worlds.
Over the last ten years, technologies such as artificial
intelligence, machine learning (ML), virtual reality (VR) and augmented reality
(AR) have all advanced at a remarkable speed. At the same time, innovations in
optics, display engineering, microprocessors and rendering algorithms have
helped to create increasingly sophisticated immersive environments. Today,
people can explore a wide range of virtual spaces, whether they use a headset
or not, from videogames and virtual tours to social platforms and educational
tools.
While VR and AR technologies (usually bracketed under XR for
extended reality) have developed independently of AI/ML, they are increasingly
converging.
“AI can enhance XR in numerous ways,” says Dr Takeshi
Kurata, Director, Research Institute on Human and Societal Augmentation, AIST
in Japan, and the Acting Chair of the joint technical committee between ISO and
IEC, ISO/IEC JTC 1/SC 24,
responsible for work on computer graphics, image processing and environmental
data representation. “AI helps create, maintain and update virtual content at a
far lower cost than manual methods,” he adds.
For example, when re-creating real‑world environments, human
appearances, or objects, generative AI can deliver hyper realistic results at a
fraction of the cost of conventional methods. The caveat is that unless checked
by human expertise, this can sometimes lead to hallucinations.
How AI can elevate XR
A growing field is augmented
intelligence - using AI to enhance human capability rather than
replace it. When paired with VR or AR, augmented intelligence could transform
creative and industrial workflows.
Nearly every XR application requires onboarding or
instructions. Instead of delivering training outside the immersive environment,
XR can embed it directly into the experience, much like videogames that teach
players through interactive play rather than manuals.
“People learn better when practice is interactive,
contextual, safe, and supported by intelligent guidance,” says Donggil
Song, Associate Professor of Engineering Technology and Industrial
Distribution at Texas A&M University. “One of the clearest examples is the
creation of virtual assistants capable of understanding natural speech and
offering personalized guidance.” This approach can be extended far beyond
application tutorials. When combined with AI techniques such as natural
language processing, sentiment analysis and pathfinding, VR becomes a powerful
platform for education and skills development.
“XR already gives us immersive learning or training
experiences, but AI gives us intelligence inside that experience,” says Song,
whose work at EinBrain Lab focuses on combining AI with XR and data‑driven
feedback to improve learning and human performance. EinBrain’s flagship project
is AlgeVerse, a fully VR, gamified college algebra platform that uses AI-based
peer mentors to give the learner guidance, feedback, and suggestions. The
project, which is in year one of a three‑year cycle, supports natural
communication and is expanding into a multi‑user environment. “Users might not
be able to differentiate which avatars are real students and which are AI peer
mentors,” Song explains.
The lab has also created a playful VR environment called
Prehistoric Protocols to make learning about computer networking more engaging.
“Students travel to a [virtual] prehistoric world with cavemen and dinosaurs.
Students must teach them using foundational IT concepts,” Song describes.
AI-powered characters in video games
The video-game sector is one of the earliest fields to adopt
practical AI in real products. Modern games use AI for world generation,
pathfinding, data analysis and player‑experience modelling. One of the most
intriguing recent developments is the population of virtual environments with
non‑player characters (NPCs)
and AI agents. “As it becomes more difficult to distinguish whether entities
are controlled by humans or AI, it may eventually become unclear whether a
human society actually exists inside the environment at all,” notes Kurata.
“From this perspective, I believe AI-driven NPC control is one of the most
impactful developments in this field.”
In rural areas of Japan facing depopulation, there are places
where large numbers of scarecrows are placed outdoors to ease the
loneliness of declining communities. “In a sense, NPCs can be viewed as a
virtual counterpart of such scarecrows,” he suggests. “However, NPCs are
controlled by AI in the backend, making them far more realistic and interactive,
even though their existence is virtual compared with physical scarecrows.”
Challenges include computational resources
Increased adoption of 5G connectivity will accelerate
AI-driven XR applications. With high bandwidth, low latency and the ability to
connect vast numbers of devices, 5G enables new forms of edge and cloud
computing. This could offload heavy processing from headsets, allowing them to
become lighter, cooler and more power‑efficient while still delivering richer
experiences.
However, integrating AI into VR and AR is not without
constraints. AI workloads often demand substantial computational resources,
which increases power draw and heat, potentially making headset devices heavier
or less comfortable. Research is underway to create more efficient AI‑specific
chips and to run AI on low‑power processors, but as AI applications grow more
complex, the tension between capability and efficiency is likely to persist.
Kurata says, “Often we see evaluation indicators like
accuracy or precision. However, there are often no metrics for energy
effectiveness. If we successfully standardize such metrics for industry,
competing companies in this space may try to reduce energy consumption as much
as possible.”
Other challenges are social rather than technical. Bias is
one of the most serious. Any AI system can reflect the biases present in its
training data. Speech recognition, for instance, often performs best for
accents similar to those of the engineers who built the system—typically male,
English‑speaking developers with American, Indian or Chinese accents.
Consequently, it is essential to evaluate AI systems to avoid unintended harm
or discrimination.
The need for interoperability standards
In addition to metrics and standards relating to the energy
efficiency of these converging systems, the field urgently needs national and
international standards for interoperability purposes, according to many
practitioners in the field, including Song. “Developing for multiple headsets
is easier than before, but still requires significant work,” he says.
“Standards reduce friction and improve interoperability, safety, reliability
and quality assurance. If we have strong standards between AI and XR,
compatibility and interoperability can be solved. Then more research and more
products become possible.”
A foundational step is to agree on XR-related terminology
without which, Kurata argues, there will be no standards and no future.
Terminology is where it all starts
Kurata’s argument, outlined in an IEEE paper,
is that when concepts span multiple domains, such as AI, XR and the Metaverse,
ambiguity in terminology often leads to misunderstanding and hinders effective
collaboration. “Much like technical interoperability in software and
systems, the clarification of terminology facilitates semantic
interoperability, which serves as a foundation for cooperation among diverse
stakeholders,” he states.
The term XR is currently widely used as an expression
encompassing VR, AR, and MR. However, there is no clear consensus regarding its
origin or meaning. “XR is sometimes explained as an abbreviation for
Extended Reality, but multiple interpretations exist regarding its etymology
and formation process,” Kurata writes in the paper XR is XR: Rethinking MR and XR as
Neutral Umbrella Terms published earlier this year.
He suggests that XR functions as a “neutral symbolic label”
by encompassing multiple “reality”-related terms. Stable usage of such
terminology requires governance through collaboration among academia, industry,
and standardisation organisations, he insists.
What standards are in the pipeline?
Kurata points to two XR-related standards currently under
development that have a strong relevance to AI. The first, ISO/IEC AWI 26073,
covers spatio-temporal mixed and augmented reality experience description
(MAR-ED) for interactive playback
“MAR-ED is highly compatible with AI because it represents
experiences semantically through meaningful events, interactions, and narrative
structures rather than only raw spatial or visual data. This enables AI agents
or AI avatars to understand, adapt, and dynamically interact with immersive
experiences, including adaptive playback and real-time branching. In this
sense, MAR-ED can also serve as a semantic experiential framework for AI-driven
XR and metaverse applications.”
Related work categorized under document ISO/IEC CD 25767 specifically
focuses on avatar face representation for XR communication. It aims to
standardize the human-to-3D avatar face modelling process, tailored for users
of XR glasses. Kurata says this work is highly relevant to AI because
realistic avatar face generation and real-time facial animation rely heavily on
AI-based face tracking, expression recognition, and generative modelling
technologies.
As both technologies increasingly merge, more standards will
be required. IEC and ISO are on the case.