“Finding Pandora X:” Reimagining Immersive Theater for VR

NAB

Live theater has been deeply affected by the lockdowns, but VR technology has created an opportunity to widen the industry's audience accessibility.

https://amplify.nabshow.com/articles/finding-pandora-x-reimagining-immersive-theater-for-vr/

‘Finding Pandora X’ by VR director Kiira Benzing, illustrates how VR can be an ideal venue for immersive theater. The show, inspired by the Greek myth of Pandora’s box, won Best VR Immersive User Experience at the 2020 Venice International Film Festival and SXSW’s Audience Award in the Virtual Cinema Spotlight category last month.

Described by creators Double Eye Studio as a multi-person interactive experience, Pandora X invites audience members to play a role in the story, interacting with the actors and the in-game world to help progress the story online.

The project has had good reviews, with Indiewire calling the experience “an unpredictable blast.”

It’s far from the first meshing of XR performance and VR theatre and certainly won’t be the last. The medium also allows for non-traditional storylines to be explored in a new way. The reach of a virtual world is one of its virtues. You can get more bums on seats than the capacity of a physical theatre.

Benzing believes the concept will become its own sub-genre enabling performances by theatre groups to expand to audiences anywhere.

“It would allow artists the means to work together at a distance in real time in a way they otherwise might not be able to afford due to budget constraints,” actor Hilary Walker tells VRScout. “It could serve as an additional platform for in person meetings/rehearsals, as well as give artists and designers a chance to walk through design elements or blocking concepts within a 3D model without being in the same room.”

Performing Pandora

Behind the scenes, the actors wore HP Reverb headsets to control their animated characters in realtime. That meant dealing with tech challenges like glitches and latency, with performers often having to troubleshoot issues live.

“They have to deal with technical hurdles thrown at them quite often, while they are performing live,” says Benzing. “They could be in the middle of a scene and they get disconnected from the internet. However, [the experience] also places them at the cutting edge of this field. These actors have become their own technologists.”

Pandora actor Pamela Winslow Kashani has a slightly different take, emphasizing the peculiar nature of acting via an avatar.

“In VR you have the difficult task of not only using the acting part of your brain but simultaneously conjuring your inner technician,” she says. “There is an element of mask work or puppetry where you embody a being who is not you and you try to discover its own truth within yourself.”

Both Venice and SXSW festivals were also virtual with SXSW doing its best to replicate the look and feel of being in Austin via a series of virtual worlds developed by French studio VRrOOm using the VR social platform VRChat. 

Extending reach

“Like all of VRchat, the entire space was accessible via PC, but in the third person; headsets provided a more immersive first-person option,” reviewed IndieWire’s Eric Kohn. “Participants can hang out afterward at a virtual bar, where I discovered attendees from Italy and Jordan — further proof of the potential for VR programming to engage global audiences.”

While SXSW has not released attendance figures for its virtual spaces, the idea is unlikely to die along with the pandemic. Kohn thinks SXSW and other festivals “would do well to maintain its presence [in the VR-sphere] and expand its brand into an arena eager to embrace its programming.”

That’s provided there’s a solid curated experience behind the showy exterior. The SXSW virtual environment provided a nifty backdrop, “but it would have been an empty technological exercise without the program assembled by XR programmer Blake Kammerdiener,” says Kohn.

Black Imagination

Another benefit is XR’s potential to amplify Black or other minority voices in the arts. An article at HP quotes Lauren Ruffin, co-founder of Crux, an organization that centers Black voices via XR storytelling. Crux recently launched Black Imagination, a series of short VR plays directed and produced by Black creatives.

Ruffin says VR can be a tool that helps underrepresented creators tell stories — and make a living doing so.

“We believe that live performance is at an impasse, that in a changing world burdened by exclusionary systems and emboldened by new technology, a seismic change can take place,” declares Black Imagination’s mission statement. “We envision a world where Black creators are being platformed on an unprecedented scale. This will transform the XR sector as we become the leaders and innovators of a revolutionary new industry. Black storytelling, Black imagination can redefine the possibilities of performance and narrative, embracing XR.”

Shakespeare in XR

It’s not just theater at the fringes which is taking a leap into XR as a way of extending the boundaries of performance and participation. In the UK, the esteemed Royal Shakespeare Company was behind an experimental version of A Midsummer Night’s Dream with the actors performing live using motion capture suits. The performance could be watched live on a PC.

Dream was billed as the first play to feature live performance capture rendered in Unreal Engine. It was performed with seven actors in a specially created 7x7meter motion capture volume created at the Guildhall in Portsmouth, southern England. The performance space includes an LED backdrop which displays the unreal world allowing performers to see their place and act within the virtual environment.

Vicon motion capture cameras and facial rigging captured the movements of the performers.  This in turn drives the virtual avatars of each of the characters in real-time through a traditional performance lighting desk into Epic Games’ Unreal Engine. The live performance was mixed with pre-recorded animation sequences.

The audience could act as fireflies guiding Puck through the virtual forest at key points in the play using the movement of a touchscreen, trackpad or mouse. The actors perform and respond to audience interaction and direction making each performance unique, as the audience will behave differently at each event.

“What’s brilliant about Dream is the innovation at play,” said Gregory Doran, RSC Artistic Director. “An audience member sitting at home influencing the live performance from wherever they are – that’s exciting.  It’s not a replacement to being in the space with the performers but it opens up new opportunities. By bringing together specialists in on-stage live performance with that of gaming and music you see how much they have in common.”

It all sounds fantastic but was far less than the sum of its parts in reality. I paid £10 to participate in Dream in March 2020 and was in equal parts frustrated and bored. Frustrated by an inability to actually influence anything on screen (using a button on a keyboard and trying to discern which ‘firefly’ in the virtual world was yours was nigh impossible). Bored, because the story was wafer thin, the visuals so dark and vague as to be uninformative and it also felt a bit pointless. I’d rather have sat back and enjoyed a more traditional telling of the play.

Of course, this was R&D and also one that was revised at the last minute from being an in person and online live performance to just being a virtual one on account of the pandemic.

But if theaters are going to go down this route and expect to make money doing so, the experience in my experience, needs to go up several notches.