Heads up on AR

AV Magazine

article here

The ‘metaverse’ is the buzzword on everybody’s lips from Silicon Valley to broadcasters like ITV even if the term itself remains open to interpretation. Most people understand that it points to the evolution of the internet as a digital world that closely mirrors our own, an experience enhanced by being three dimensional and persistent across application and device.

The chance to access it has given the makers of virtual, augmented and mixed reality (VR/AR/MR) displays fresh impetus. A new wave of smart glasses is to hit the market over the next 18 months promising a more comfortable experience for wearers than earlier devices, such as Meta’s Oculus Rift, Google Glass and Magic Leap One – all discontinued but with new versions in the works.

“Wearing bulky headsets on your face is not going to be the future of the metaverse,” declares Michael Hayes, CEO at InWith Corp.

“Great big bulky glasses are uncomfortable and you can’t deploy them,” agrees Paul Travers, founder and CEO of Vuzix which is trying to perfect an ‘Oakley-style fashion-forward’ device.

CEO at Korean startup LetinAR, Jaehyeok Kim believes smart glasses are the only device that can combine the metaverse and the real world and will eventually “change the fundamentals of gaming, of business and of our daily lives.”

How smart glasses work
Most smart glasses work using optical waveguides. These are structures imprinted on a piece of glass (either one or both lenses) that bends light to the eye from a source (usually a mini-projector) embedded in the corner of the frame. The source overlays virtual objects on the wearer’s vision with data typically derived from a smartphone. Recent developments have tended toward enabling wireless connectivity for the smartphone and reducing the size and weight of the headgear.

Only a few manufacturers can produce plastic AR optics and LetinAR claims it has the best performing version around. “Plastic fabrication of smart glasses using injection molds is very cost-effective and scalable to mass production,” explains LetinAR’s Kim.

The company has also developed a patented concept called PinMR that enables manufacturers to pair waveguides with OLED microdisplays. Kim claims this technique consumes less power and has a smaller form factor than other light sources.

LetinAR wants to license its technology to manufacturers and expects to announce a couple of partners by the end of the year. Adds Kim: “We hope to see our solution be the first one to mass-produce and change how people see the world.”

Mixed Reality headgear
There are currently two fundamental types of AR headsets: see-through and passthrough. Most current AR headsets, such as HoloLens or Magic Leap, are see-through. You view the real world directly, with a layer of virtual objects superimposed on to the glass (or projected into your eye). The field of view is narrow and virtual objects cannot be fully opaque.

Passthrough uses outward-facing cameras to blend computer graphics with a video feed of the real world, supposedly creating a more immersive, mixed reality experience across a much wider field of view.

France-based startup, Lynx has launched a version costing around $1,000 primarily aimed at enterprise users but major tech giants are poised to deliver MR headsets capable of VR and video passthrough AR at consumer prices.

They include Meta’s Project Cambria a wireless headmounted display which includes eye tracking. That not only gives Meta more data about the user’s gaze but it will enable ‘foveated rendering’ which is a more efficient way of processing just the graphics needed in a field of view. The product, which could debut this year, also features face tracking to allow in-game avatars to mimic the player’s facial expressions in realtime, potentially even opening up accessibility functions, such as lip reading.

Meanwhile, Google is keeping details of Project Iris more of a secret but it is being overseen by Clay Bavor, the same executive managing Project Starline, an experimental 3D telepresence system. It’s not thought to be launching until 2024 when it will likely compete with similar devices from Microsoft, Samsung and Apple.

The enterprise
Analysts such as ABI Research expect the consumer market for smart glasses to grow 100 per cent year on year between now and 2026, (when 28 million smart glasses will ship) but it is applications in the corporate sphere which have led growth to date. The most active industries using MR headwear are military, manufacturing and healthcare.

Peggy Johnson, Magic Leap CEO summed up the reason nicely in a Bloomberg Magic Leap Targets Medical Customers for Its Augmented Reality Headset – Bloomberg interview. It’s easiest to make inroads with “industries that are used to wearing something on their eyes.”
Vuzix’ Travers supports this. “You can solve specific problems targeting the enterprise by not having to be a general purpose consumer device.”

Vuzix corporate solutions stem from the Tac-Eye monocular weapons sighting system it developed for US special forces in 1997. Its M400C glasses are being used today to train surgeons in medical procedures and by shift workers for more efficient picking of parts in warehouses (and to prevent theft).

The company’s latest development, Shield features some breakthrough technology. Explains Travers: “Most activities for which you’d wear AR glasses are conducted at arms’ length but normal waveguides are set at infinity. That means when you put a 3D image in front of someone’s eyes to appear 2ft away there is a focus problem that causes stress in a lot of people.

“We’ve designed our waveguides to change the focal distance to converge about 1.5m away so it makes the experience much more comfortable.”

Vuzix has coupled this with another proprietary technology capable of emitting two million nits of light from microLEDs the size of a grain of rice. “Waveguides are relatively inefficient in terms of light transmission,” says Travers.

“You’ve got to output millions of nits to make up for that fact. The best OLEDs are pushing thirty thousand nits. MicroLEDs shrunk to the micron level and packaged into a projector smaller than a pencil eraser means you can put it in a pair of glasses and it practically disappears (from view).”

Travers claims that this is a world first. “Nobody is doing microLEDs one micron in size. You get a fraction of the volume required, a contrast ratio of one million to one, and crystal clear optics.”

It only displays monochrome. “For barcode scanning of inventory or delivering instructions on what valve to adjust there’s no need for full colour,” he explains, though a colour version is in the works.

The much anticipated Magic Leap 2 is expected to launch this year. Twice as powerful as the first generation, Magic Leap 2 also features a greatly expanded vertical field of view, 18 built-in cameras and sensors to enhance real-world tracking and dynamic dimming to make it possible to see the colour black and view 3D content in brighter environments. The tethered device also has a sleeker and more wearable form factor.

Microsoft’s third version of HoloLens is being lined up for launch in 2024. In an interview with the Wall Street Journal (‘How AR Will Change the Way We Work’), Microsoft’s head of mixed reality, Alex Kipman stated that this next generation device would “not just be an incremental improvement but a transformative leap forward.”

Smart biology
With technology included in smart glasses already at the nano-level the logical evolution is to eliminate the headset all together. Connected contact lenses point one way forward.
California firm, InWith Corp holds patents in embedding component circuitry into soft hydrogel (rather than hard gas permeable) contact lenses and has tested them with lens maker, Bausch and Lomb.

CEO and co-inventor Michael Hayes is seeking developers to license the “smart biology.”

“Our technology has the dual capability to tune a wearer’s eyesight (to enable bifocal short and long distance focus) and to augment vision with graphical information just like a heads-up display,” he says. “Contact lenses will be much more discrete and comfortable than having to wearing smart glasses.”

It is powered by micro battery, connects to the user’s phone via Bluetooth and has a patented method of harvesting energy from the blinking of the wearer’s eyes.
Information displayed could be health related (such as monitoring blood sugar levels) or functions familiar to a smart watch like a gyroscope.

Hayes also points to real world alerts such as speed limits and restaurant discounts and, eventually, full immersion.

A commercial launch needs to clear US health regulatory hurdles. Hayes thinks that this is not an issue but says the lens is not quite ready to be brought out of stealth mode.

5G and Wi-Fi 6E connectivity
With AR still most likely to be powered by the smarts of mobile phones, the rollout of 5G with faster speeds and lower latencies is considered critical for enabling realtime interactivity between a user and virtual information/graphics being streamed live.

Like 5G, the latest upgrade to Wi-Fi promises faster wireless speeds and lower latencies in addition to less signal interference for networked devices to create a more stable and reliable connection. Wi-Fi 6E (which operates in the 6GHz band) supports internet speeds of greater than 1Gbps. However, users are going to have to have devices capable of connecting to 5G C-Band and Wi-Fi 6E – which could either boost or stall AR growth.

Embryonic metaverse and AR device development
The metaverse of augmented reality and virtual reality software, and the hardware with which we will access it, remain in the foothills of development.

“We’re starting to talk about the metaverse now the same way we started to talk about the internet in the early 1990s,” said Steve Koenig, vice-president for Research at the Consumer Technology Association (CTA) at a press conference for the Consumer Electronics Show. “(At that time) when we were all using 56k modems and dial-up internet, it was impossible to imagine the things we’d be doing online today. I think the same is true with the metaverse.”

Koenig pointed to the building blocks which will be used to assemble “a metaversian immersive digital experience”. These include cloud computing, haptics, volumetric video and 5G networks.

Insists LetinAT’s Kim: “AR glasses are the ultimate form of the metaverse platform because they help interact with the real world and the virtual world. Still, it will take multiple years before the world sees the true form factor of AR glasses as people imagine in the movies.”
Higher performance (even wireless) battery tech and superior chips from makers like Qualcomm and Nvidia will advance to enable smaller, lighter and more powerful virtual experiences.

“Down the road you won’t need a phone just a pocket device that is connected,” says Travers. “(The tech industry) will take all the high compute that is needed in AR glasses right now and put it in a device that is in your pocket or on your wrist.”

Longer term we may not need a conventional wearable at all. A microchip embedded into our nervous system could enable humans to communicate with the internet by thought alone.

Elon Musk-owned lab Neuralink is taking the brain-machine interface out of science fiction. The company is designing a neural implant that will let individuals control a computer or mobile device “unmediated and in high fidelity”, according to the company https://neuralink.com/

The idea is to insert micron-scale electrodes into areas of the brain that control movement.

“The Neuralink app would allow you to control your iOS device, keyboard and mouse directly with the activity of your brain, just by thinking about it,” says the company.

It is about to launch clinical trials with humans.