VIRTUAL WORLDS
The week of June 28, 2015

Walking a mile in someone else’s shoes

By Selena Larson

Marcelle Hopkins wants to make you feel what it’s like to watch your child die from malnutrition.

“Hunger itself is very visceral, and this kind of technology—virtual reality—people often have a very visceral reaction to it,” Hopkins said in an interview. “When we thought about how to convey what is happening in South Sudan, we realized that this technology could pair very well with a story. The technology could serve the story.” Through VR, Hopkins can tell a story from the perspective of residents in South Sudan.

Her documentary, Recipe for Famine, chronicles South Sudan’s ongoing humanitarian crises stemming from a civil war that began in 2013, including an exponential rise in food insecurity. Since December 2013, more than 1.6 million people have been forced to flee their homes, and 10,000 people have been killed, according to the Council on Foreign Relations. But she’s not creating the typical talking-heads documentary that asks viewers to empathize with strangers at a distance, a world away, visible only through a screen. She’s using virtual reality to take you into the heart of the catastrophe—to not only see the tragedy but feel it.

Strap on a pair of Oculus Rift VR goggles, and you will be transported to a village where crops refuse to grow and food and water are scarce. Instead of just watching someone struggle to live on a day-to-day basis as a displaced person, you will be placed in their shoes. The point of view shifts dramatically, inserting viewers into a 360-degree view of a landscape they’ve never seen before.

Recipe for Famine is among a handful of VR projects designed to evoke empathy by letting filmgoers inhabit a situation they wouldn’t otherwise know. At the Sundance Film Festival this year, Rose Troche debuted her film Perspective, which presents the story of a sexual assault from both a man and a woman’s point of view. Project Syria provides a first-person view of the ongoing Syrian civil war, with bombs detonating around an overflowing refugee camp.

“The technology could serve the story.”

In a review of Perspective, BuzzFeed reporter Adam B. Vary said, “I felt trapped, forced to live through something I never would find myself living through, at least on the guy’s end … Is the friend a predatory creep, or just really young and tragically reckless? — we are clearly not meant to walk away from The Party thinking what ultimately happened was anything other than rape.”

perspectiveA promotional still from Perspective. 

These works can provoke a uniquely empathetic response. Because VR gives our brain visual and auditory inputs that make us feel like we are experiencing something from a first-person perspective, pieces like Recipe for Famine heighten our emotional and empathetic response, according to Jayde Lovell, neuroscientist and host of the YouTube channel Did Someone Say Science?

“The more visual cues we have, the easier it is for our imagination—just watching someone, most people have the ability to put ourselves in their shoes. But it’s not quite the same as experiencing it from their perspective,” Lovell said in an interview. “[Virtual reality] basically gives our brain more inputs than they would otherwise have, and makes it easier for our brain to fill in the rest of the scenario.”

Now that VR has become so much more realistic, viewing a life-like immersive game or film can have greater impact, both on our mental comprehension of the film itself and our offline behavior.

Stanford’s Virtual Human Interaction Lab researches the effects VR has on our real-life interactions. One of its immersive studies showed participants what it’s like to experience the world with color-blindness, and participants in the VR experience spent more time helping a color-blind person after the test than those who simply imagined what it would be like.

“Several of our studies show that manipulating aspects of a virtual environment can affect the perception of empathy in that environment.”

“Several of our studies show that manipulating aspects of a virtual environment can affect the perception of empathy in that environment, or affect your own empathetic behavior,” Shawnee Baughman, lab manager at VHIL, said via email.

Another Stanford University study used a superhero theme to research whether becoming a life-saving avatar in a simulated environment would lead to increased altruism. Participants either embodied a superhero or became a passenger in a helicopter and were tasked with saving a diabetic boy or simply flying around the city. After the simulation, researchers purposely knocked over a set of pens and judged people’s reactions. Those who played the superhero role helped pick up the mess more frequently than those who remained in the helicopter.

“Our brains are not evolved yet to refuse the convincing visual stimuli, so even though you know the environment is not ‘real,’ your brain and body respond as if it is real,” she said. “So, you can create an experience in VR that will change a person the same way a real experience might change them. Other forms of media do not afford this type of experiential learning.”

Right now, virtual reality studies are focused on the short term, examining the effects immediately after a VR experience or perhaps a week or two later. But Stanford is working on longer-term studies, and as technological advances have brought VR to customers via the Oculus Rift, Samsung Gear VR, and Google Cardboard, researchers see new opportunities for study.

While VR is having something of a consumer renaissance, scientists have experimented with it for decades. For almost 20 years, scientists have researched how people with autism can learn empathetic behavior through VR simulation. A 2013 study demonstrated how VR positively affected emotion recognition and social and occupational functioning. Dutch company IJsfontein received funding from the Netherlands government to come up with a tech-based simulation to better prepare family and friends to care for someone with dementia through improving empathy. Instead of creating it for a VR headset, IJsfontein and a team of researchers and academics built a one-hour simulation called Into D’mentia that literally walks participants through an average day in the life of a patient.

“Even though you know the environment is not ‘real,’ your brain and body respond as if it is real.”

The simulation takes place in a container built to mirror a typical apartment of an elderly individual. The participant wears a vest that speaks the inner monologue of the patient as well as instructions on getting around the apartment. On the kitchen table, a projection of photo books and different papers simulates difficulties keeping track of memories and to-do lists; at the back of the apartment, a “daughter” is projected on a screen, and she yells at participants for forgetting something.

“People really get into it and really feel the alienation and the loneliness of someone with dementia,” Naomi van Stelten, commercial concept manager for IJsfontein, said in an interview. “It’s really confronting, and since you’re all alone, it’s a really tense experience.”

Into D’mentia

Many filmmakers want to produce that tense experience—to confront the audience with a raw representation of experience. But creating a film for VR is unlike traditional shooting techniques. Directors have to give up a measure of control in the film, because at any point in time, viewers can turn their head and see something happening behind, rather than in front, of them. The film doesn’t cut to different camera angles—the viewer does.

To capture a full immersive experience, filmmakers must shoot in 360-degrees, meaning there are lenses coming out in all directions, recording every possible viewpoint. Once filmed, that footage must be stitched together into a sphere and mapped onto the inside of a sphere, Hopkins explained. This process can take a while, especially if camera angles don’t match up or the edges of the lenses aren’t clean. Hopkins told me that she’s working on a camera setup that will facilitate a better stitching process—though she wouldn’t go into details on what the crew is experimenting with.

Another challenge is determining from which perspective to tell the story. Hopkins said the film will be guided by a few different subjects who live in South Sudan, explicitly avoiding narration from the film’s correspondents in order to compound the feeling of literally being in someone else’s shoes.

Creating a film for VR is unlike anything done before with traditional shooting techniques.

“At this point for me because it’s all new it seems infinitely more complicated than traditional narrative filmmaking,” Hopkins said. “The technology is innovating every day, and there are things that are available now that weren’t available a month ago, and I know by the time we start post-production, I hope that other things will be available then, better tools to work with.”

Because immersive VR is still so groundbreaking, Hopkins and her team must go through a number of trials-and-errors to figure out the best methods to producing a film. It will be a learning process for everyone—one Hopkins plans on blogging about throughout the making of the film; she also hopes to open-source a VR production toolkit that includes any code written for the film.

Hopkins and her team, which includes Evan Wexler and Andrew Blum, will be joining the film’s producer Ben Moran in South Sudan in early August to begin filming. Until then, they are testing and perfecting the camera equipment in New York.

In the end, Hopkins hopes that viewers will leave the film feeling compelled to learn more about the crises in South Sudan and that awareness and education will allow individuals or nonprofits to respond with calls to action. She also hopes that the documentary can be a roadmap for future immersive journalism projects.

“I love learning new things, it’s one of the reasons that I got into journalism in the first place—I’m a very curious person and I love learning about new things, new ways of working, and new worlds,” she said. “To be working in a medium that does not yet have rules is very freeing in a way, and it’s also kind of scary. We have very few people to tell us how to do it. I know we are going to make a lot of mistakes, and that’s okay. Because we hope other people will learn from our mistakes.”

Photo via Matthew G/Flickr (CC BY 2.0) | Remix by Max Fleishman