Video and photos by Tim Trainor and Jeff Collet
What do the visual memories of someone who is blind look like?
That’s the question driving Mary Anne Funk’s terminal project for her master’s degree in multimedia journalism.
Funk, who expects to graduate in 2020, is developing an interactive 3D sound installation and tactile experience to share the memories of Abby, an Ethiopian woman living in Portland who was bit in the eye by a cobra when she was 8 years old.
Though now completely blind, when Abby dreams, she sees the colors, places and people she remembers from her childhood. The virtual world Funk hopes to build will recreate the sights and sounds that still exist in Abby’s mind’s eye.
Funk plans to use spatial audio — a surround-sound technique that makes 360-degree video more realistic — to put the audience into that moment and place.
To make her vision a reality, Funk is using immersive experience technologies in the Oregon Reality (OR) Lab, an interdisciplinary space dedicated to the teaching and research of immersive media as an ethical tool for social innovation. Her final project will include head-tracking earphones, listening stations with triggers to play audio stories, and spatial sounds collected from Ethiopia. Stations will include braille for people who are blind, and tactile photographs will allow visitors to “see” with their hands.
“I’ve got some learning to do before I get there,” said Funk, a visual ethnographer with 20 years’ experience as a documentary filmmaker and photojournalist. “But I want to know how I get to the point where I can build that.”
Making — and researching — the media of the future
Launched in January by the School of Journalism and Communication at UO Portland, the OR Lab is a testing ground where students, faculty researchers and Portland community members can use and develop virtual-, augmented- and mixed-reality media to solve social and business problems. Students have been taking classes, such as Immersive Storytelling: 360 Video and Introduction to Social Virtual Reality, in the lab since winter term, and faculty are conducting research there.
The OR Lab is outfitted with the latest in immersive reality equipment and standing experiences that incorporate whole rooms. It houses editing suites where students can create 360-degree video, practice photogrammetry and use state-of-the-art software.
“There is magic in this space that can transport you to another place and experience,” said Donna Davis, director of the OR Lab and the strategic communication master's program.
Davis has spent more than a decade researching social virtual media, particularly in the virtual world Second Life. Her research on embodied experience and identity in virtual reality among people with disabilities was funded by a grant from the National Science Foundation. She is now expanding her research to study the ethical implications of immersive media.
“As communicators and educators, we've got to inspire our students and engage industry to think about how to ethically use these technologies to tell our stories, build communities and create relationships in them,” Davis said. “If we don’t, this emerging technology could evolve without a conscience.”
Funk’s multimedia journalism master’s classes introduced her to the basics of 360-degree video, photogrammetry and augmented reality. Since then, she has been diving into spatial audio.
Students in other programs are also taking advantage of the virtual lab. Recent graduate Fnan Berhe centered her strategic communication master’s thesis, “Building Experiences with XR Technology: A Guide for Storytellers of Today,” on how virtual reality will change the communication industry. Berhe said her time at UO helped her get ahead of the trends, and she believes the OR Lab is vital to helping future cohorts do the same.
—Mary Anne Funk, Documentarian/Photojournalist, Multimedia Journalism Master’s Student
Doctoral students are also using the lab for classwork and research. And undergrads in the School of Journalism and Communication will soon be able to pursue similar immersive media projects and research in the school's new Experience Hub, which opens on the Eugene campus this fall.
Research topics are as varied as 3D reconstruction, character animation and accessibility issues in virtual reality. Media studies master’s graduate Spencer Bennett focused his 2018 thesis on the benefits of virtual reality meditation.
Bringing innovation to the community
The OR Lab is not just for students and faculty. Davis said Portland-area communication and virtual reality professionals are looking for a place to practice and perfect immersive media techniques. That’s why the lab’s location at the UO Portland campus is ideal.
“The city is definitely a hub, a center for really innovative work that's happening in virtual, augmented and cross-reality experiences right now,” she said.
Future evening, weekend and summer offerings in the lab will be designed for working professionals and community members who want to understand these new technologies and learn how to harness them. Public workshops will explore the evolution of social virtual reality, using immersive platforms for communication, and building content for virtual environments.
UO Portland students from the Oregon Executive MBA, architecture, historic preservation and sports product design are also planning to use it for their own projects in the future.
Davis believes immersive and augmented media are where social media was 10 years ago — the next frontier for the communication industry. She said students and professionals who want to stay ahead of the curve should check out UO’s new OR Lab.
“As a format for storytelling, community building, experiential marketing — there are so many ways immersive media can be used right now,” she said. “And we want to start poking and testing and building and playing in all of those spaces.”
(Source: Statista)
(Source: Greenlight Ventures)
Immersive media glossary
There’s a full spectrum of immersive media technologies, and the field is constantly evolving. Here are a few terms describing some of the tech UO students are learning about in the OR Lab:
360-degree video:
To make these immersive videos, videographers record a view in every direction at the same time with an omnidirectional camera or a collection of cameras. Watch a video about multimedia journalism master’s student and OPB reporter Cassandra Profita’s production of a 360-degree video for the Eagle Creek Fire project.
Augmented reality (AR):
Augmented reality adds to a user’s vision by inserting digital information and images into what they see naturally. For instance, a person using AR may see a new image, animation or video layered on top of a view of the room in front of them. This technology is used in popular games such as Pokemon Go!.
Mixed reality (MR):
Also called extended reality or hybrid reality, mixed reality combines physical reality and digital content in a way that enables interaction with and among real-world and virtual objects.
Photogrammetry:
This process digitally stitches together overlapping photographs, using a software program like RealityCapture, to create a 3D model or map. Watch a video explaining how multimedia journalism students used photogrammetry to map a virtual world.
Spatial audio (ambisonics):
This surround-sound technique mimics the way we hear in real life by channeling the characteristics of sound as it travels through space and time.
Virtual reality (VR):
Virtual reality technologies replace reality and the user’s vision and hearing with a new digital world. It transports them through their audio and visual senses to worlds and experiences only previously imagined. This can take place on a screen but is most commonly experienced with VR headsets. Examples are screen-based virtual worlds like Second Life, VR games like Beat Saber and Lone Echo, and social environments like AltSpaceVR and Sansar.
High tech tools in the OR lab
To make and enjoy virtual reality, you need a lot of gear. The Oregon Reality Lab has all the latest, including (from top left):
1. Camera sensors
2. Oculus Rift headset and controllers
3. HTC Vibe headset and handset
4. GoPro battery
5. UO notepad for jotting down ideas
6. 360-degree cameras from Garmin, Samsung and GoPro
Photo by Tim Trainor
