University of Southern California, Annenberg is one of the 2017 winners of the Challenge Fund for Innovation in Journalism Education. See all 10 winners.
Team
- Robert Hernandez, Professor, USC Annenberg
- Caitlin Shamberg, Senior Digital Editor, KCRW
- Kevin Tsukii, Immersive Video Lead, Emblematic Group
- Francois Bar, Professor, USC Annenberg
Learn about some of USC Annenberg’s JOVRNALISM (VR Journalism) students here and Hernandez’s 360 projects here.
Describe your project
While we’ll use VR (360 video, photogrammetry, videogrammetry, etc.) to capture and tell stories of a local community to the local community, the real goal of this project is to explore how we might translate these high-tech stories into easily accessible community experiences. Whether through pop-up VR spaces in libraries, presenting at community events or offering video projection domes, we want to find ways for the larger local community to experience these innovative pieces. We will cover both local history and current issues, and we want these stories to be social experiences that aren’t limited to affluent users with VR headsets.
What is your live news experiment?
We will produce immersive stories about communities in Los Angeles — for example, Leimert Park, which has been called “the last battleground” facing gentrification — and present the pieces through various means that translate the high-tech to low-fi. Ideas include pop-up VR viewing stations at libraries, community centers or galleries; partnering with community events; and organizing screenings.
In addition to building spaces where people can individually try on head-mounted displays (HMDs) like HTC or Oculus, we want to explore projection domes that allow several people to experience the project at the same time.
We aim to use the latest technology available, like photogrammetry and videogrammetry, to produce pieces ranging from the rich local history to today’s battles against gentrification. This new technology brings to life oral history and testimonies, allowing us to produce holographic interactions that offer real presence and connection with the subject.
If the experiment works, what do you think might happen?
Los Angeles is quick to forget its own history. This project will explore how a community might preserve and tell its history in a new, impactful medium. By doing so, viewers will understand what might be lost as the neighborhood faces gentrification. At the same time, these stories could bridge older and younger generations as they provide context to better understand the community.
While the journalistic goal is community conversation and a better understanding, the technological goal is experiencing the immersive projects in social settings. If we don’t engage groups or showcase the content at social events — and are limited to one-offs — then we have failed. If it’s just a story that only a handful of people experience, we have failed. We hope the experiences have a lasting impact on the local conversation by engaging diverse voices in the debate about our community.
How is this project unique and innovative?
We want to push our current technological capabilities. We already have a good handle on creating 360 videos and VR experiences, but we want to be among the first news experiments that explore photogrammetry and videogrammetry, which creates 3D models based on photos and videos.
This production will also use spatial audio to create a more realistic presence with the subjects and stories. Imagine meeting someone through a realistic hologram and feeling their presence and voice as they tell you their story. (Blackout, which debuted at Tribeca, and USC Shoah’s Dimensions in New Testimony are recent examples of this technology.)
We will also experiment with innovative presentations of this content outside of a singular headset experience. While we will certainly use HMD setups, we want to explore video domes and interactive big screens, allowing for multiple people to experience pieces together. If the budget allows, there are viewing setups which synchronize HMD-style viewing we can test out, but the open group experience is our goal.
What technology platforms will you use?
While we know Kolor, Mettle’s Skybox and Adobe Premiere produce immersive 360 videos, we will explore emerging setups to produce photogrammetry and videogrammetry. Kits we are currently testing range from Occipital’s Structure Sensor to DepthKit by Smile. We have also been in talks with Microsoft Hololens and the LA-based startup 8i, which are higher end videogrammetry studios. These experiences will then be inputted into Unity to create volumetric experiences that can be exported into immersive videos or interactive apps, depending on the final goal.
If it works, how might this experiment change teaching at your school or media practices in your partner’s newsroom?
While there are several potential takeaways — including defining best practices in VR innovation — the main takeaway is how to foster community engagement when producing high-end immersive stories that are often consumed alone and limited to audiences with HMDs. Also, this project helps build a growing partnership between USC Annenberg and KCRW, as we both invest in building relationships with a diverse community that is threatened by change and gentrification.