Face Off: “Welcome to Chechnya” Brings AI to Civil Rights Fight

 NAB Amplify

When the Academy announced its Oscars shortlist for VFX it had pundits scrambling to make sense of one of the nominees. Out were Wonder Woman 1984, Tom Hanks’ naval drama Greyhound and Marvel spin-off The New Mutants. In were Tenet, Love and Monsters, Mulan… and Welcome to Chechnya, a documentary detailing the brutality the LGBTQ community faces at the hands of the Chechnyan government. 

https://amplify.nabshow.com/articles/face-off-welcome-to-chechnya-brings-ai-to-civil-rights-fight/

The film may not have made it to through the bake-off but retains the distinction of being the first ever doc to be shortlisted for consideration in this category. It’s also to the Academy’s credit that it recognised the achievements of work a long way outside the mainstream.

In investigative reportage when the identity of subjects need to be protected they are conventionally blurred out or appear in silhouette.  That’s even more the case here when the queer Chechens are in danger inside and outside of the country from what has been described as ethnic cleansing.

Filmmaker David France was all too aware of his obligations but also wanted to preserve the emotional resonance of their experiences. So with VFX supervisor Ryan Laney, they struck upon a ground-breaking combination of digital face-replacement and machine-learning software to replace some of the individuals in the film with digital stand-ins.

They’re have been a multitude of face-swapping CG in recent blockbusters including Thanos / Josh Brolin in Avengers: Endgame but this is journalism where the integrity of the story really matters.

Laney describes the technique as “a bit like a prosthetic where the new face is painted over the old face, so it’s 100 percent the original person’s motion, we’re not fabricating anything new.”

France and Laney, a VFX guru who has worked on Fight Club, Jurassic Park III, Harry Potter and the Chamber of Secrets and Ant-Man, trailed several ideas including rotomation — of the kind seen in A Scanner Darkly  — to standard issue pixelated faces to asking artists to reinterpret the faces, only to find this superimposed an artist’s impression on the reality.

They even tried a Snapchat-like technology to put digital glasses, masks or new noses to disguise them in some way.  “What that really wasn’t doing, though, was helping us tell this really urgent, human story,” France and Laney tell Digital Trends.

“We kept losing the human aspect of it. It wasn’t until we saw Ryan’s first pass on the face swap using a volunteer that we knew we had something that would allow us to show the movie to an audience. We had promised everyone in the film that we wouldn’t release it until they were satisfied with their disguises and their presentation.”

France asked 22 people — mostly queer activists in New York — to lend their faces as a physical shield to protect the people in the film. They were filmed against green-screen from multiple angles in varied lighting conditions with their faces subsequently mapped using machine learning over the subjects in the film. 

“While all eye and mouth movement and facial tics belong to the original subject they are all being carried out beneath the skin of these volunteers,” France says. “This could allow us to know their stories; it’s still them. We see the weight on their faces. It comes through, it’s unmanipulated, we’re picking up those expressions.”

The filmmakers also went to extraordinary lengths to capture original footage on location in Chechnya, including over writing the cards rather than just deleting the data in case it fell into the wrong hands.

In the U.S in postproduction “we built a secret lab that was entirely offline, so all of our turnovers were on a hand-delivered drive that was given to me in person. All of the transfers we did for dailies and work reviews were all done in very encrypted fashion with passwords that weren’t shared online.”

Every frame was scrutinized to insure nothing that could put a subject at risk was removed. “We knew it was going to be studied on a forensic level,” France admits.

R&D alone took the best of a year. And budgeted at just $2.2 million the film includes 400 shots, or an hour of its 107 minutes run time. 

“That’s a big change for documentary filmmaking. There’s now this tool for filmmakers to tell their stories in ways that haven’t been done before, and it also provides some additional security for witnesses to tell their stories and do it in a human way. They don’t need to be monsters in the shadows. They can have a voice and be in the light and have their story translated effectively and truthfully.”