“Star Trek” and the Strange New Worlds of Spatial Computing

NAB

article here 

Imagine stepping into the Holodeck, a concept once confined to the realms of Star Trek and the visionary mind of its creator, Gene Roddenberry. That future is edging closer to reality, according to Jules Urbach, founder and CEO of OTOY, who will showcase the latest developments in the technology at NAB Show.

These breakthroughs are “a major step towards realizing that goal,” Urbach says. “This is an exciting inflection point, and we are just at ground zero.”

Urbach’s cloud graphics company has teamed with the Roddenberry estate over many years and, more recently, with Apple, to deliver new 3D and interactive experiences for users of Apple Vision Pro, and will share their findings at the NAB Show session “Boldly Go: Star Trek’s Voyage in the Age of Apple Vision Pro.” Part of the Core Education Collection: Create Series, the session will be moderated by entertainment industry futurist Ted Schilowitz from 3:00-4:30 PM on Tuesday, April 16 in Rooms W210-W211.

Urbach grew up in Los Angeles with the Roddenberrys as neighbors, and he is best friends with Gene’s son Eugene “Rod” Roddenberry, the CEO of Roddenberry Entertainment. Urbach says it is been a long-term vision of his not only to recreate Star Trek experiences with full visual immersion, but to build the tools that will enable spatial content creation experienced through headgear and, eventually, as holograms in a holodeck.

“One of the fundamental messages I want to talk about at NAB is that there is a long term plan to find a way to create the tools that will eventually render a holographic display inside of a room — and we will get there,” Urbach says.

Urbach says was inspired to start OTOY by Jon Karafin, who has vast experience in holographic engineering, including at specialist camera developer Lytro and as CEO of Light Field Lab. Karafin — who will also appear on the panel at NAB Show — is already commercializing panels capable of holographic display. Urbach was so impressed that OTOY invested in Light Field Lab.

“You could cover a wall with these panels today as a step toward the holodeck. This is the future,” Urbach says. “Costs will come down and technology will advance.”

At NAB Show, Urbach and Roddenberry will present OTOY concept videos and documentary films from recent Roddenberry Archive releases, which have been remastered specifically for the Vision Pro. These include unique interviews with George Lucas and Stan Lee exploring Gene Roddenberry’s influence on Star Wars and Marvel.

“There is nothing else like it out there,” Urbach says of the Vision Pro. “The quality of ray tracing and resolution is incredible. And this is just Version 1. Just think of where we are now with the iPhone 15. There is so much potential.”

He points to the Universal Scene Description format as an important standard for building the ecosystem of spatial content creation. OTOY is a member of The Alliance for OpenUSD — a group formed by Pixar and Apple — which is working to standardize the USD format across the industry to help artists and developers create and deploy complex real-time 3D experiences at scale.

Featured in the Roddenberry archive’s native spatial content for Apple Vision Pro are 1-meter by 1-meter light field cubes that are displayed at 90-frames-per-second in 4K resolution per eye, pushing the boundary on visual fidelity.

“We can scan an actor in 3D, storing that spatial content in a mezzanine format then render at any resolution. This could be to the Vision Pro or a holographic display; the output is different, but the fundamentals are the same. You can move through the video seeing light and reflections with fully path traced real time lighting, including dynamic glossy reflections and shadows, bringing new levels of photorealism to immersive content. This is what we have cracked.”

Urbach recalls interviewing his friend’s father, at age 12 for the school newspaper. “It’s been a dream come true to digitize all of Gene Roddenberry’s inspirations, interviews and concept artwork into the archive,” he says.

“There are things in the archive that blew me away, such as the notes that Gene shared with [science fiction writer] Arthur C. Clarke.”

When the original television series was canceled in 1969, Roddenberry lobbied Paramount Pictures to continue the franchise through a feature film. He was encouraged to pursue this by Clarke, who had seen his own short story made into 2001: A Space Odyssey by Stanley Kubrick in 1968.

“Without the inspiration and encouragement of Arthur C. Clarke, the continuation of Star Trek might not have happened,” he explains.

Urbach, Roddenberry and Karafin will be joined by Richard Kerris, NVIDIA’s head of Media & Entertainment, to highlight how diverse technological advances — ranging from light field displays and virtual production to generative AI and decentralized computing — are transforming the media and entertainment industries.

When considering where we are today with spatial entertainment experiences and discussions about the holograms — which are no longer hypothetical — it is worth recalling Arthur C. Clarke’s Third Law: “Any sufficiently advanced technology is indistinguishable from magic.”