Above: Elden Ring boss battle, the kind of game content that’s very difficult to create with Gen AI (i.e. the most impactful part of Elden Ring)

Originally posted on my Patreon

I recently wrote about how 3D artists tend to hate Gen AI for proprietary reasons (i.e. copying artists’ work and style without their consent); now let’s spend some time looking at the business productivity case for the technology.

Does Gen AI enable faster, more cost-effective creation of 3D content in online game worlds?

Maybe in some cases, at some point in the future.

But probably not for most cases right now.

Demo of an AR Custom Landmarkers project that Francis created for Snap — see more on his YouTube

At least that’s the argument I’d make, especially after talking with Francis Chen, a veteran real-time 3D artist who’s created mobile-based augmented reality experiences for major companies like Snap, HBO, and Paramount. (Longtime SLers may also remember him as “Francis Chung”, creator of the famed Dominus Shadow muscle car, and other classic virtual world innovations.)

One key challenge of using Gen AI on a professional basis, he explains, is the complexity and revisions that come with creating most 3D game experiences:

“If you’re making a big game,” as Francis explains, “AI makes more mistakes the more revisions you need to make in a game, since it can’t handle complex problems well yet. It’ll cost more to clean up the mess.”

Think, for example, of a spectacular boss fight in a big budget AAA game, where there are thousands of interactions with on-scene graphics in every single fame.

“[O]nce you combine physics, animations, VFX, different fight moves, that’s where it’ll get messy. You need more creative control in making sure what you’re doing matches what the game designer wants.... and with AI, you’re often times putting more prompts to match the complexity.”

And the thing is, more complexity leads to even more complexity.

“Generally AI gets lost the more levels of creative-decision making are required in different parts of game design,” as Francis puts it. “So you might get a fight scene generated by AI that is 50-60% there... but players are paying for 100%.” (Especially when players are expected to pay upwards of $70 for a major game -- hence much of gamers’ animus against any perceived corner cutting that “cheats” them with Gen AI.)

Still, if prompting can get us 50-60% there, does that mean it’s a useful starting tool? Maybe:

“That’s a case-by-case scenario, but in some cases, yes,” Francis tells me. “AI shouldn’t be replacing workflows but rather supplementing existing game development pipelines. It depends on which game development specialization [it’s being used for]-- you’ll have more subjectivity with art, versus programming.”

That also leads to another hidden inefficiency with using Gen AI:

It adds an extra layer of decision-making into the virtual world development workflow. I.E. “Should we use Gen AI for this specific content we need, or not?” Baldur’s Gate 3 creator Larian Studios, for example, tested Gen AI tools in creating game content, only to find out the results were substandard, and junked the test as a waste of time. (After already spending a lot of time in testing.)

“That’s why I’m too cautious about AI, especially in game dev which is very revision heavy,” Francis tells me. “It can streamline certain game development workflows (i.e. ‘making a character controller for a monster following a fighter’), but the large efficiency gains the business executives want from AI are very different from what AI can do right now.”

Which brings up a final, paradoxical challenge of Gen AI:

The very fact that 3D content can be generated from a simple text prompt could make the task less efficient, not more: Since it’s so easy to prompt up 3D content, the temptation will be to prompt up lots and lots of it... and then spend a lot of extra time reviewing, editing, and deleting most of it. (That’s part of the problem that caused Rec Room to start sliding last year -- too much Gen AI slop on their servers.)

“Especially those long prompts,” Francis concurs. “Google Gemini gives me, like, paragraphs.”

Which is also why Francis is talking with me for this article at all: It’s been a challenge trying to communicate the problems with Gen AI to owners and managers of big game studios. Since they don’t develop 3D content themselves, they don’t see the setbacks they will encounter, by embracing Gen AI too hastily.

Join my Patreon to read stories like this first!