EA partners with the company behind Stable Diffusion to make games with AI

[tronn4@lemmy.world]

Double boycott

[duamerthrax@lemmy.world]

I don't know what they're taking about. Amarillo's Butt Slapper isn't published by EA.

[jjjalljs@ttrpg.network]

LLM generated code is notoriously bad. Like, "call this function that doesn't exist" is common. Maybe a more specialized model would do better, but I don't think it would ever be completely reliable.

But even aside from that, it's not going to be able to map the free form user input to behavior that isn't already defined. If there's nothing written to handle "stand on the table and make a speech", or "climb over that wall" it's not going to be able to make the NPC do that even if the player is telling them too.

But maybe you're more right than I am. I don't know. I don't do game development. I find it hard to imagine it won't frequently run into situations where natural language input demands stuff the engine doesn't know how to do.

[fonix232@fedia.io]

My own fucking experience. Which I've already explained in detail above.

[v_krishna@lemmy.ml]

It's very obvious in this thread that you have hands on experience and many others do not. 20+ years professional SWE here, a majority of it applied ML/big data/etc. LLMs are really bad at many things but specifically using them as a natural language layer over NPC interactions would be relatively easy and seems like a great use case honestly.

[fonix232@fedia.io]

Okay I won't even read past the first paragraph because you're so incredibly wrong that it hurts.

First generation LLMs were bad at writing long batches of code, today we're on the fourth (or by some metric, fifth) generation.

I've trained LLM agents on massive codebases that resulted in <0.1% fault ratio on first pass. Besides, tool calling is a thing, but I guess if I started detailing how MCP servers work and how they can be utilised to ensure an LLM agents doesn't do incorrect calls, you'd come up with another 2-3 year old argument that simply doesn't have a foot to stand on today.

[sugar_in_your_tea@sh.itjust.works]

So, repeatedly buy and return games?

[jjjalljs@ttrpg.network]

lol if you had read the rest of my post you would have seen I admitted you might be right. But go off, I guess.

[beetus@lemmy.world]

So what's your process and how is it a boon?

[beetus@lemmy.world]

Great for you. You did say "almost every workflow". How many workflows exist beyond your own lived experience? Do you work on games, do you know all the workflows there? Citation absolutely fucking needed.

[beetus@lemmy.world]

If I wanted to talk to NPCs at length I'd just type into a fucking chat bot. I play games to experience the developers story and vision, not endlessly prompt an npc.

I think your experience will be a common one for players.

[fonix232@fedia.io]

Alright I did read further and damn, you just keep going on being wrong, buddy!

Yes, you can fucking do "stand on the table and make a speech" work. You know how? By breaking it up into detailed steps (pun intended), something that LLMs are awesome at!

For example in this case the LLM could query the position and direction of the table compared to the NPC and do the following:

• plan a natural path between the two points (although the game engine most likely already has such a function)
• make the NPC follow that path
• upon path end, it will instruct the NPC to step onto the table via existing functions (Skyrim pretty much has all these base behaviours already coded, but the scripting engine should also be able to modify the skeleton rig of an NPC directly, which means the LLM can easily write it)
• then the script can initiate dialogue too.

I've asked Perplexity (not even one of the best coding agents out there, it's mistake ratio is around 5%), and within seconds it spit out a full on script to identify the nearest table or desk, and start talking. You can take a look here. And while my Papyrus is a bit rusty, it does seem correct on even the third read-through - but that's the fun part, one does not need trust the AI, as this script can be run through a compiler or even a validator (which let's be honest is a stripped down compiler first stage) to verify it isn't faulty, which the LLM can then interact with and iterate over the code based on the compiler feedback (which would point out errors).

now mind you this is the output of an internet-enabled, research oriented LLM that hasn't been fine-tuned for Papyrus and Skyrim. With some work you could probably get a 0.5B local model that does only natural language to Papyrus translation, combined with a 4B LLM that does the context expansion (aka what you see in the Perplexity feed, my simple request being detailed step by step) and reiteration.

You'd also be surprised just how flexible game engines are. Especially freeroaming, RPG style engines. Devs are usually lazy so they don't want to hardcore all the behaviours, so they create ways to make it simple for game designers to actually code those behaviours and share between units. For example, both a regular object (say, a chair) and a character type object (such as an NPC) will have a move() function that moves them from A to B, but latter will have extra calls in that function that ensure the humanoid character isn't just sliding to the new position but taking steps as it moves, turns the right direction and so on. Once all these base behaviours are available, it's super easy to put them together. This is precisely why we have so many high quality Skyrim mods (or in general for Bethesda games).

And again, code quality in LLMs has come a VERY long way. I'm a software engineer by trade, and I'd say somewhere between 80-90% of all the code I write is actually done by AI. I still oversee it, review what it does, direct it the right way when it does something silly, but those aren't as minor functionalities as we're talking here. I've had AI code a full on display driver for a microcontroller, with very specific restrictions, in about 4 hours (and I'd argue 2 of that was spent with running the driver and evaluating the result manually then identifying the issue and working out a solution with the LLM). In 4 hours I managed to do what otherwise would've taken me about a week.

Now imagine that the same thing only needs to do relatively small tasks, not figure out optimal data caching and updating strategies tied to active information delivery to the user with appropriate transformation into UI state holders.

[facedeer@fedia.io]

They have every opportunity to tell me they're not liking what I run. Turns out I know them better than you do.

[pycorax@sh.itjust.works]

Respawn still made some pretty fun games. But errr yea that's pretty much it.

[jjjalljs@ttrpg.network]

Yes, you can fucking do “stand on the table and make a speech” work. You know how? By breaking it up into detailed steps (pun intended), something that LLMs are awesome at!

My intended point was the LLM at run time taking user input wouldn't be able to do "make a speech" if the game engine doesn't have that concept already encoded. And if the game is presented as "take user input and respond believably" then users are going to ask for stuff the engine can't do. Maybe there's no animations for climbing. Maybe they did some shortcuts and the graphics look bizarre when stuff is elevated.

I wasn't talking about Skyrim specifically.

But also you're being unpleasant in this exchange, so you can win.

[facedeer@fedia.io]

Major applications I use AI for:

• Art, obviously. I generate lots of illustrations of the characters and other things they encounter. These days image generators have become good enough that I usually just have to describe things well in the prompt and get usable results, but if I need to edit the images to get them closer to what I need I run ComfyUI locally and have lots of models for that as well.
• I use producer.ai to generate custom music. Sometimes it's diagetic - in the previous campaign the party encountered an NPC that loved to sing about important plot developments in-universe - but mostly it's either background soundscape or supplementary songs (consider them like "theme music" that plays during the intro or credits). Producer.ai closed down new subscriptions a while back but I've also used Udio and Suno and they're both good too.
• I record the sessions and transcribe them using WhisperX. I feed the transcripts into NotebookLM and have it generate summaries and notes about the events of each session. It can also generate "overview" videos that work great as a 5-minute catch-up players can watch before the session if they've forgotten what happened last time.
• Since NotebookLM has all my campaign notes in it as source documents, as well as PDFs of the rulebooks, it's great for quickly whipping up stats for creatures when the players do something unexpected. It's also been a good brainstorming assistant, and it "knows" the names of every random little NPC or village we've seen along the way.

I've been experimenting with the Wan2.2 video model lately. It's not quite up to snuff for generating videos of meaningful length, but it's still pretty neat to be able to put a character portrait in and have a 5-second snippet of them just "being alive." I think it'll be a neat addition to having static portraits.

Aside from producer.ai, all of these tools are free. Though the WhisperX transcription program I use is a custom Python script, I'm not sure what would be a good solution for a non-programmer to spin up.

Edit: Downvoting me isn't making any of these tools less useful.

[facedeer@fedia.io]

I haven't been getting downvoted as much as I used to these days, at least. I think more people have started to actually experiment with AI and see what it's good at, and are finding it to be not the Great Satan that popular opinion paints it as.

[dogs0n@sh.itjust.works]

Will my steam account get banned if I pre-order every one of their games, click play for 5 minutes then instantly refund? I have my faith, but I don't know if they'd see that as abuse of their refund policy.

[dogs0n@sh.itjust.works]

For them

[nucleative@lemmy.world]

Carry on, sir.