GPT characters in games

Submitted by YearZero t3_126rcq0 in singularity

This is what I really want. Generating worlds from prompts is cool. But I’d love to see more intelligence in game characters. Maybe within certain parameters of the story and personality and role of the character, but with freedom to improvise. Imagine playing a Skyrim type game or Cyberpunk 2077, but have a different experience each time? Maybe you can make certain characters into friends or foes based on how your conversation goes.

Maybe the characters can have decision making based on the same principles. A character can follow you, attack you (and decide exactly how, with what, and to what degree), or convince its friends to join it for the attack etc.

I think this should be the next real frontier in gaming. Graphics are already pretty damn good. Physics certainly could use some work but it’s coming along. Destructible environments and such are hard but also not always needed for every game.

But AI has been stuck in decision trees for decades. We aren’t making clear progress in game character AI like other stuff, and we need a proper leap. Games would become way more engaging.

The question is - can modern CPU and GPU handle all the inference, especially with multiple characters all interacting at the same time? While also handling the rest of the game physics and graphics. I’d sacrifice the latter for an experimental attempt at the former.

73

Comments

You must log in or register to comment.

genericrich t1_jeahu1l wrote

I don't think it will be useful for games, since games are storytelling medium and introducing randomness that can't be well-controlled into stories makes them into bad stories.

I don't think it will be feasible to work well enough.

−9

Mortal-Region t1_jeaia9q wrote

>We aren’t making clear progress in game character AI like other stuff, and we need a proper leap.

Problem is, it's such a huge leap going from decision trees to autonomous agents who can form their own objectives and plans with respect to other characters and the environment. It's pretty much the same problem that brains evolved for. LLMs aren't agents, so I think they'll end up as automated dialogue generators, with the overall storyline still being "on rails".

6

3z3ki3l t1_jeakjny wrote

Replying to genericrich (#2,490,318)

But it could be used for background characters. That way it’s not the same line every time you run into a character.

So every guard in Skyrim didn’t take an arrow to the knee. Could be as strict as generating some other injury, or loose enough to provide an entirely different excuse for not joining the war.

15

genericrich t1_jeakxai wrote

Replying to 3z3ki3l (#2,491,035)

Sure, if you can control it. The problem is that the developers need to be very sure that they aren't introducing a bug or meandering, meaningless, dead-end side quest. That would be hard to verify, IMO, with an AI-generated content layer.

I guess we'll find out sooner or later, because I am sure these will be rolling out soon.

0

3z3ki3l t1_jeanw6e wrote

Replying to genericrich (#2,491,125)

Eh. Skyrim already has generated side-quests. It creates characters for you to go kill, and items for you to steal. I don’t think a little bit of dialogue, whether it fits perfectly or not, would break that much.

3

WonderFactory t1_jeaotp1 wrote

I'm actually adding ChatGPT NPCs to the Unreal Engine 5 game I'm developing at the moment, it's a rogue like set in a post singularity world, gameplay is similar to Hades so there's plenty of dialogue in the game. at the minute it's difficult to get a model to run on the local PC so I'm using OpenAI's API. There are challenges like latency while you're waiting for the API to return, it's also quite expensive so releasing a free demo of the game is out of the question. It could potentially cost several dollars per user in API fees over the life of the game so that will of course limit your pricing flexibility, you can only reduce the price so much in steam sales etc. I'm hoping though that inference costs will come down by the time the game is finished.

I haven't posted any footage with the GPT dialogue added to the game but I might post it here in a couple of weeks.

28

WonderFactory t1_jeap8lf wrote

Replying to genericrich (#2,490,318)

I think it can work, I'm trying to get it working in a game I'm developing at the moment. You have to have a mix of randomness and structured story telling. I literally have to say to ChatGPT, the user is saying xyz, reply to the user but try to work this plot point into your reply.

3

sumane12 t1_jeb1wh1 wrote

Replying to WonderFactory (#2,492,056)

>I'm hoping though that inference costs will come down by the time the game is finished.

Genius. This is exactly what will happen, and I'm glad someone has the forethought to develop the product, before the underlying technology is ready, because it will be there.

13

megadonkeyx t1_jeb4ecc wrote

It's almost inevitable that it's going to happen, in a few months you won't be able to take a leak without an AI telling you to wash your hands. (Exaggeration)

Anyhow, technically doing this on the local machine is the way to go, checkout the 7b llama models that can be quantized down to 4bit. Something like that trained for games, run it on the background on a single thread.

It would not be super fast but it could chug away in the background analysing the game state and making decisions.

4

MassiveWasabi t1_jeb55ha wrote

Check out this paper that Microsoft researchers just released. Among a ton of other cool things, they talk about how this new model they are working on, TaskMatrix.Ai, will be able to take control of "AI teammates" in team-based games, and that you can give each individual teammate different tasks in order to carry out a complex strategy. This seems like the next step to having truly dynamic characters controlled by AI, hopefully so dynamic that they seem completely real.

10

psdwizzard t1_jebn0zq wrote

Replying to WonderFactory (#2,492,056)

you may be able to run a custom version of Llama instead of using GPT. You could even train in on GPT3.5. Just tell GPT its a character from the game, build your data set from its answers and use it like Alpaca does.

5

genericrich t1_jebntv6 wrote

Replying to koa_lala (#2,498,506)

Uh, ok. (Old man grumbles about having worked at two AAA studios and several small game companies, including one he co-founded. Started in QA and ended as lead game designer before leaving the industry for more lucrative work.)

1

No_Ninja3309_NoNoYes t1_jebobu3 wrote

If we all sign the LAION petition, we will get whatever we want. But someone will disrupt the game industry soon for sure. I hope they get rid of the text prompts too. Why can't we just upload some rough sketches instead? Or other files.

1

ReignOfKaos t1_jebvoyd wrote

Replying to Mortal-Region (#2,490,427)

In theory, reinforcement learning could solve this, but specifying the reward function is very hard and if you need human evaluation then it is very slow. I think it’s not quite as theoretical of a solution as evolutionary algorithms, as it’s been successfully applied to create very capable game playing agents before, but I think it’s very difficult to beat behavior trees or simple utility systems when it comes to creating characters that are fun to play with.

2

2Punx2Furious t1_jecz2b2 wrote

Replying to WonderFactory (#2,492,056)

I think you could get around the latency issue by having the generated dialogue come in form of letters that you receive in-game, which would feel a lot more natural than a slow conversation. Or have some cutscenes in between the prompt and the answers. As for the price, it should probably be an optional setting, and maybe the price should be offset by a subscription or ads, as much as I hate them, but in this case it would be difficult to do otherwise, unless you plan to foot the bill of your users forever.

1

azriel777 t1_jed9mfh wrote

Replying to Mortal-Region (#2,490,427)

Some combo system. The NPC's are like skyrim, on rails with some randomness so it gives the illusion of being alive, but when the user interacts with them, the AI takes over and fleshes them out to give them life and when you leave them they revert back to what they were, perhaps with some modifications to their rules and behavior depending on what the user did. The AI could also randomly take over NPC's to generate interesting things for the user to see or generate quests for the user. The potential is there, just going to take some trial and error to figure out what works and doesn't.

2

FoniksMunkee t1_jedcqe9 wrote

Replying to [deleted] (#2,515,104)

It's interesting - I work in AAA games and work with a lot of clients. They are deathly silent on any kind of AI integration so far. So it will be interesting to see when this starts coming down the pipe.

3

FoniksMunkee t1_jediwl9 wrote

Replying to [deleted] (#2,516,919)

Yes, that's a good point I didn't think of is API costs.

The other issue is games kinda move slowly in some respects. There are games that started before ChatGPT was commonly known that wont be finished until after ChatGPT 5 is out. And there is no way these tools will be integrated during that process.

They would also have to convince Sony, MS and Nintendo to have their SDK's in a model. I don't think MS will necessarily have a problem with that... but there's a ton of third party libraries that would need to come on board, not to mention how you deal with existing legacy code. There are still more companies in the AAA industry with custom engines than are using commercial engines like UE4/UE5.

Then comes the RND, then comes the new game... so what, 5 years before we see wide spread adoption in the AAA market?

2

FoniksMunkee t1_jedm0mj wrote

Replying to [deleted] (#2,518,641)

There will be teams that are starting games today - that might look at ways of using it. I Know of some teams using it for art pipelines (but mostly inspiration / mood boards). And there will be some tools like Photoshop and Blender etc that will mean we will see it appear in the pipelines earlier.

But as far as wide spread penetration - it will probably need to wait for teams to start new projects. So obviously, a team starting today may choose to do something already. But widespread use? It's way too risky. It's hard enough to justify an upgrade from UE4.27 to UE5.1. I can't imagine up-ending an entire gameplay system just to integrate AI would be an easy sell.

And while I won't be the first to know when it starts happening... I will be early on because of where I sit in the pipeline. All I hear right now is crickets.

1