The future of performance: Real-time technology in film and theatre

Real-time technology is revolutionising the entertainment industry, transforming both film and theatre in ways previously thought unimaginable. At the forefront of this transformation is virtual production, a method that allows filmmakers to shoot on digital sets with real-time adjustments. This offers a more flexible and cost-effective alternative to traditional filming techniques. The technology, most prominently featured in the Disney+ series The Mandalorian, is changing how stories are told on screen.

The Mandalorian made history by utilising Stagecraft, a cutting-edge virtual production platform developed by Industrial Light & Magic (ILM). Stagecraft combines vast, high-quality digital environments with physical sets, enabling filmmakers to shoot scenes that appear to take place in exotic locations without leaving the soundstage. This groundbreaking technology allows the instant modification of virtual environments, significantly reducing the need for location shoots or expensive physical sets.

Jon Favreau, the series’ creator, explained that his experience on The Jungle Book and The Lion King convinced him “there had been breakthroughs in game-engine technology that were the key to solving this problem.” Favreau recognised that the real-time capabilities of these tools could fundamentally change the filmmaking process by allowing directors to visualise and alter their scenes live on set. The result is a seamless fusion of physical and digital elements that removes many of the constraints of traditional production.

Artificial intelligence is also reshaping the creative landscape of filmmaking. According to The Guardian, ScriptBook’s AI primarily analyses scripts rather than relying on cast or director data. “Most people believe that cast is everything, but we’ve learned that the story has the highest predictive value,” said Nadira Azermai, the company’s founder. Within six minutes, the software can assess more than 400 parameters, including emotion analysis, protagonist and antagonist journeys, structural elements like the three-act format, and the potential appeal to niche or broad audiences. While the inclusion of cast details can enhance accuracy, Azermai notes that “if the computer says no to begin with, no additional information will change it to a yes.”

In post-production, AI continues to push boundaries. Adobe’s Sensei AI, integrated into Adobe Premiere Pro, assists video editors by automating tasks such as trimming and sequencing footage based on mood, pacing and rhythm. AI handles much of the technical workload, freeing editors to focus on storytelling and creative direction. This acceleration of the editing process enables films to be completed on tighter timelines without compromising quality.

Theatre has also embraced real-time technology. Companies are increasingly adopting digital innovations such as augmented reality (AR) and virtual reality (VR) to enhance live performances and create immersive experiences. The Royal Shakespeare Company (RSC), for instance, integrated AR into its 2016 production of The Tempest, in collaboration with Intel and Imaginarium Studios. This production featured a dynamic digital environment that responded to the actors’ movements in real time, including a digitally rendered storm that transformed the stage experience for the audience.

The innovation has extended into immersive formats as well. In 2021, the Royal Shakespeare Company launched Dream, a digital theatre project based on A Midsummer Night’s Dream. The production used motion capture and gaming tools to build a computer-generated forest. Actors moved inside a performance space, and the system mapped their actions onto avatars in real time. Audiences joined remotely and played a role in shaping the world. This project opened a door to new ways of experiencing theatre.

Finally, live-streaming continues to be a powerful tool in expanding access to theatre. The UK’s National Theatre has launched a new global streaming platform, making its acclaimed productions accessible to viewers around the world. The initiative opens up new revenue opportunities while democratising access to high-quality theatre, allowing audiences to experience powerful performances beyond the physical stage.