Let’s kick this off by taking a page out of a certain South Park episode that did the unthinkable: it was wholly created by artificial intelligence. With the use of AI in movies and television having been a heated point of contention during the prolonged Writers Guild of America strike, and on the heels of a tentative agreement between the WGA and Hollywood’s motion picture producers, I expect some of you will get upset with me saying the future of entertainment remains one where AI will handle many creative tasks, reshaping the industry. That isn’t science fiction. It’s a reality we should accept and embrace.
In the past, our machine-driven characters would plod along with heavy pre-programmed behaviors and continuous prompts. But players in the futuristic world of entertainment soon will be able to improvise their own scenes to meet high storytelling standards and goals crafted by AI. What we need to realize, though, is that this isn’t all dystopian doom and gloom. There’s potential here for a shift in roles and responsibilities, not just job obsolescence, a host of opportunities for true collaborations between human and machine creators. Today’s writers could become tomorrow’s AI whisperers, and animators might evolve into AI art directors.
Here’s the how and why.
The Wit and Wisdom of … Bots?
In their research paper titled To Infinity and Beyond: SHOW-1 and Showrunner Agents in Multi-Agent Simulations, the powerhouse developers at San Francisco-based Fable Studio (a virtual and augmented reality company) propose a new method of AI content generation and lay out the groundwork for a futuristic Hollywood–one where decentralized AI teams, acting as both writers and animators, carry out the commands of higher-level AI showrunners. Their cutting-edge tech absolutely has the potential to revolutionize creative processes like writing, animation, and visual effects–but the question of the hour is whether this will lead to artificial entertainers who can truly grasp comedy, tragedy and the full spectrum of human emotional experience. Can bots bring down the house with wit and wisdom rather than cold calculation?
Like crafty card sharks at the casino, the SHOW-1 team is stacking the deck in their favor. But maybe that’s not the best metaphor. What they’re doing shares more with the efforts of choreographers and theater technicians working in synch to stage a major Vegas dance routine, clearing obstructions and balancing blocks with an aptitude that would fill any Busby Berkeley fan with awe. And in case you don’t know it, Berkeley was an American film director and musical choreographer who devised elaborate musical production numbers that often involved complex geometric patterns.
Anyways, the researchers have designed two types of AI agents, calling them, respectively, Showrunner Agents and SHOW-1 agents. The first play the role of directors, dictating abstract, broad-stroke goals and creative constraints without meddling in minutiae. They’re the Steven Spielbergs of the AI world. The SHOW-1 Agents are essentially worker bees, or actors, who fulfill the goals set by the Showrunners. The ragtag band of SHOW-1 thespians begin their journey with no more autonomy than puppets on marionette strings. But through trial simulations they gain a taste for thinking on their feet.
To get a clear sense of this picture a warehouse full of movable boxes. The Showrunner dictates the desired box configuration as a creative goal, and the SHOW-1 agents independently get to work, figuring out policies to rearrange the boxes efficiently through deep reinforcement learning–a learning based on rewards. Their only guide — cryptic pixel patterns conjured by the Showrunners on avant-garde vision quests. It’s a bit like a group of improv actors working together to create a scene.
Over time, running the agents through these simulations, and then recording observations about their behavior, generates training data. Each SHOW-1 agent has been trained independently, but knowledge can be transferred between agents, and collective behaviors emerge through multi-agent interaction. In addition to boxes, some experiments introduced helper and obstruction agents to increase complexity. Again, Showrunners can constrain available actions–placing limits on what the SHOW-1s can do–to further shape emergent behaviors.
The results demonstrate that decentralized SHOW-1 agents can learn policies to effectively rearrange boxes into patterns provided by Showrunners, both individually and in teams. Basically they act like the cybernetic Borg from Star Trek, sharing a hive mind and acting together as one, but with individual personalities that can provide character depth when it serves the director’s intentions. The innovators at Fable Studio propose applying all this to continual learning scenarios with adaptive Borgs, ahem, I mean … agents … that can learn and expand upon their skills.
So, What’s All This Mean for the Entertainment Industry?
In terms of writing and story generation, it potentially opens up vast new creative landscapes. Imagine someone feeding plot points to an AI showrunner and letting a team of AI agents fill in the details like dialogue and scene descriptions. Suddenly a single flesh-and-blood person could command an entire digital writers’ room, allowing and establishing high-level creative goals along with themes, desired arcs and outcomes. Teams of AI agent policies could then be trained to construct the intermediate details — dialog, descriptions, etc — that progress towards those goals.
Now consider the realm of animation and visual effects. By instructing swarms of AI agents, Showrunners could automate complex tasks such as character movement, environment modeling, and asset creation. Though there’s still a long road ahead to match the emotional depth and creative nuances of human artists, we will eventually get there, and it’s going to revolutionize the industry.
If you have a few minutes, I’d ask that you check out the AI Agent/South Park example I mentioned at the top of this piece. You can watch it in full or just give it a quick scan. Either way, I’ll be here and we’ll meet up on the other side.
OK, I see that you’re back. Great! I like having company here. And I’ve been quietly watching the animation along with you.
Now let’s talk. It’s important to bear in mind that the simulation was only a proof-of-concept. But it still wasn’t all that bad, and I even had a few chuckles watching it. You can’t help but notice AI’s limited understanding of sarcasm, though. And South Park without sarcasm is like a taco without hot sauce. As any good director would tell you, the magic lies in the details, the artistry. To truly capture the essence of human animation and CG artists, AI’s creativity, emotion, and expression need substantial upgrades. Until then, AI might serve best in a supporting role, taking over routine tasks, setting the stage for human artists to shine. This South Park episode was easy because of the simple style of animation, but AI visual effects (VFX) will only improve in the upcoming years.
AI’s Deepfake Implications in Video and Audio Generation
When AI meets existing deepfake technology, we could be looking at some show-stopping performances. Picture Showrunner agents defining cast, scenarios, speech content, and more, with SHOW-1agents delivering the raw video, audio, and overlay. The result? Highly realistic human-like media so compelling it could make you question your own reality.
Of course, this comes with its share of controversy. Legal and ethical dilemmas surface around consent, likeness rights, misinformation, and other things. Properly implemented, however, AI could redefine generative media, eliminating traditional costs and barriers. So what happens to our jobs? While we’ll likely see a shift in the job market, there’s no need for an all-out panic. The innovations we’re discussing could actually create new roles within the industry. Writers could transition into AI story architects, designing broad narrative structures and plot points for an artificial intelligence to follow. Meanwhile, animators might become AI artistic supervisors, shaping the aesthetic constraints for the AI and ensuring the results meet their artistic visions. Similarly, sound engineers could evolve into AI sound designers, dictating the audio landscape for an AI to generate. And let’s not forget the emergence of new positions, such as AI ethics officers, responsible for ensuring that AI-created content meets ethical standards, or AI content moderators, safeguarding against AI-generated misinformation and explicit content.
AI isn’t poised to boot humans out of Hollywood, but rather to spark a reshuffling of roles and responsibilities. It opens an exciting frontier with a lot of potential for innovation, but like any tool, it needs to be wielded responsibly. As these AI systems continue to evolve, we must ensure they’re used ethically and to the benefit of society, prioritizing human creativity and wellbeing over purely economic incentives.
So how about I get ahead of the curve and coin a new job title for myself, which I kind of did earlier in this essay. Are you ready … ?
AI Whisperer!
Nah, maybe not.
Next: Artificial Intelligence Ethics Aren’t Optional
About the author:
