Stanford researchers collaborate on new StageCast technology for performers separated due to COVID-19

Michael Rau was directing an opera in New York City when the world shut down last March. Rau, who is the artistic director of Theater and Performance Studies (TAPS) at Stanford University, returned to Palo Alto to ponder the future of the performing arts when live productions were no longer viable and technology had not yet caught up to the needs of performers.

Researchers across Stanford have created a new tool to help performers who are located in different spaces come together in online performances. (Image credit: iStock)

Soon, Rau was in conversation with Tsachy Weissman, professor of electrical engineering, and Keith Winstein, assistant professor of computer science, about the lack of tools for making performances in the virtual realm. “So the challenge I posed for them was what if we make something that’s different and better,” said Rau, an assistant professor of TAPS in the School of Humanities and Sciences.

After receiving a grant from the Ethics, Society and Technology Hub’s COVID-19 Rapid Response program, the three got to work. The trio, along with computer science graduate student Sadjad Fouladi, developed a fall course that enlisted students to design and develop a platform called StageCast for hosting a live theatrical production over the internet.

The fruits of their labors will be unveiled to the public on March 4-6 in a performance called StageCast: Experiments in Performance and Technology, which will include four short plays performed by Stanford actors in different locations.

Enter StageCast

StageCast is built upon low-latency videoconferencing technology previously developed by Fouladi, which recently was awarded the 2020 Applied Networking Research Prize.

On existing commercial platforms like Zoom, the pieces of programs that compress video and transmit packets of data have been designed separately and then combined. In contrast, the StageCast system uses a combination of custom software and purchased hardware to significantly reduce the latency experienced on other platforms. This improves the experience of both audio and video during live performances so that both singing and acting can be in synch.

“The goal is to make digitally streamed theatrical performance more real and more pleasant by reducing the delay between participants, compared with commercial systems,” Winstein said.

The Stanford team plans to make the StageCast system open source, once they have finished developing the software, in order to allow other theater artists to make digital performances. StageCast is not available to the public at this time, but Weissman said he hopes it will eventually impact not just theater, but also online teaching and other activities that involve videoconferencing as well.

Along with the use of the software, the system also involved sending actors in the plays two different cameras, lights and special microphones. “What we’re hoping to do is to make different cuts and use different angles, relying on the grammar of cinema, to create meaning and story on film,” said Rau who is serving as the director of the short plays. “We even have a green screen ability to make it look like the actors are in the same space talking to each other.”

The faculty noted that part of the fun and challenge of the project was bringing together their different disciplines. For instance, theater students would describe the functions of a lightboard to the computer science and electrical engineering students, who would then write code to allow the lights in the homes of the actors to be controlled remotely.

“It’s an unusual privilege as researchers to have this kind of direct engagement with the people affected by our work, and it taught us a lot about the challenges of real-time internet videoconferencing that the research community hasn’t been focusing on,” Winstein said.

Ethical implications of technology

In addition to technical challenges, Rau and his team also considered the ethical ramifications of StageCast. “I also wanted us to think about the consequences for industry,” Rau said. “We had conversations with stakeholders involved to make sure it’s actually useful. I didn’t want to run into ethically problematic situations and to wonder if this would end up putting more people out of work.”

Part of the project involved commissioning four professional writers, including playwrights Mike Lew, Kate Tarker, Kelley Girod and Michael Yates Crowley, to write short, 10-minute plays to be performed by Stanford students. Rau and the team also worked with professional New York musicians, including composer and Broadway performer Gelsey Bell along with violinist Josh Modney and cellist Mariel Roberts, who will perform musical interludes between the plays. This allowed Rau to get feedback from professionals in the field about how they might use the tool in the future and how they felt about creating online rather than live productions.

What he learned is that many in the industry believe that digital streaming will be part of the performing arts in some way even when live theater is allowed to resume. Some people may want to see a live performance and will put a premium on that experience, Rau said, but others may not want to drive a long distance, hire a babysitter or incur other inconveniences and expenses to see a live show.

“I imagine a future where hybrid shows – simultaneous in-person live and digitally streamed performance – become the norm for our industry, and we hope that the StageCast system can play a role in developing the digital infrastructure for a future generation of artists working in the performing arts,” Rau said.