Air Head creators say OpenAI's Sora finicky to work with, needs hundreds of prompts, serious VFX work for under 2 minutes of cohesive story ^↺

When OpenAI announced Sora, its video generator AI, one of the videos that was being used to demonstrate its capabilities is the Shy Kids short titled Air Head. While the video was initially touted as an impressive show reel for the OpenAI model, a recent FX Guide interview with Shy Kids reveals that far more work went into the video than many had thought.

While what Sora can do is certainly impressive and was nigh impossible to do just a year or two ago, the Shy Kids team still took nearly two weeks to create Air Head — mostly due to the limitations of the AI. One of the biggest hurdles Shy Kids ran into with Sora was its lack of cohesion, which forced the production team to use an unorthodox editing method, not unlike creating a found footage film or documentary.

It was just getting a whole bunch of shots and trying to cut it up in an interesting way to the VO. – Patrick Cederberg, post-production on Air Head

Shy Kids says it had a script for the video, but the team had to be fluid and adapt to the varied output generated by Sora. Shy Kids also had a hard time keeping things consistent from shot to shot, with Sora often generating a different type of head on the balloon. Overall, Cederberg says it took “hundreds of generations” to get just under a minute and a half of edited footage for the video. He also estimates that the ratio of source material to final content was 300:1, meaning Shy Kids only used around 0.33% of the video Sora generated in its final edit.

My math is bad, but I would guess probably 300:1 in terms of the amount of source material to what ended up in the final.

Working with Sora meant more than just generating hundreds of clips, though. The team also had to manually go in and perform all the regular tasks like colour grading, retiming, and even VFX to remove unwanted elements from the frame. In one generated clip, Sora output a balloon with a face imprinted on the front, and in others, the balloon would be different colours or have an unwanted string hanging from the bottom — all of which had to be removed.

More advanced applications of VFX saw the Shy Kids team removing an entire head that had been generated onto Sonny, the main character, in place of the balloon. Things like this were removed in Adobe After Effects (which is $34.99/mo. and available on Amazon) in order to reach a final product.

While Sora and generative AI video has come a long way, it seems like it's far from replacing the artists behind the scenes — especially if the content being produced is meant to be coherent or anything longer than a few seconds. This likely also explains why, with the exception of two, all the “unedited” clips OpenAI has posted to its Sora page are all on the order of 20 seconds or less.