Exploring LoRa Models in Game Development

What We Do: We use LoRa models to efficiently adapt and create visual styles in our games, keeping the process quick and the file sizes small.
Usefulness for Us: This technology speeds up our design process, enabling rapid experimentation and iteration.
Benefits for Players: Players can use this technology to create their own concept art, adding a personal touch to their gaming experience.
“LoRa models” What are they, and how can they benefit your game design process? LoRA or Low-Rank Adaptation, applies small changes to the most critical part of Stable Diffusion models: The cross-attention layers, where the image and the prompt interact.
Unlike other training methods like Dreambooth, which are powerful but create huge files, or textual inversion, which makes tiny files but is less versatile, LoRA strikes a balance. It keeps file sizes manageable (2 - 200 MBs) while still offering solid training capabilities.
Here’s how it works
LoRA introduces small, trainable components into the base-model’s architecture - allowing for significant reductions in memory usage and computational demand. LoRA allows for greater experimentation and agility in 2d concept art. You can rapidly prototype new ideas and adapt traditional AI assets to better fit your game’s unique aesthetic and narrative style.
DreamBooth Implementation
We utilize DreamBooth to adapt a general diffusion model specifically to our artistic style, using a select number of images created by our SpaceEngineers Art Team. This technique helped us integrate our unique style onto three separate base models. For the purpose of training the models, we used only the art created in Keen Software House.
Testing Base Models
We work with our concept artists to quickly visualize sketches with the application of our base models, and assess concept viability. Although this method speeds up the early prototyping phase, making these concepts fully functional still requires some extra hands-on work.
To address this, we introduce customized LoRa models.
Creating LoRas
By adding LoRas, we enhance our base models with components that serve as an emphasis onto the original style, resulting in promising outputs with just a single generation. Like with our general diffusion models, these LoRas are trained on images exclusively sourced from our concept artists, ensuring clear understanding of the artistic and technical nuances of our game.
Empowering Artists
This combination of models allow our artists to experiment with and refine potential designs efficiently, setting a precise foundation early in the creative process.
Resources
Stable Diffusion: huggingface.co/stabilityai/st...
Automatic1111: github.com/AUTOMATIC1111/stab...
ControlNET: huggingface.co/lllyasviel/Con...
Lora Model:
civitai.com/models/452399/spa...
civitai.com/models/452608/spa...

[@Ponial]

Saw Mech legs in thumbnail, got bit to exited. Ah well

[@warwizard1309]

Now I want an epic voice in all technical videos

[@DiceKnight]

I love how out of place it feels. It's like a voice for some serious milsim game like arma.

[@celltechindustries]

It's cool to see you guys try out new things, as a 3d artist myself, AI has been a very good boost to my productivity and creativity. Being able to quickly make reference images and concept art for ideas floating around my head has been invaluable, and I hope you guys keep experimenting and sharing future models. I do believe that keeping everything open source and transparent to the public is the proper way forward, so it's awesome to see the LoRA released. The comments here just prove that the everyday person just doesn't have a clue what they're talking about when it comes to AI, and at the end of the day, the majority of customers aren't gonna care if the product is good. EDIT: I do however agree, that the AI voice is a bit much and feels very impersonal.

[@LittleJohnnyBrown]

It feels just as impersonal as everything else generated does. I'm also an artist, I do both 2d and 3d and seeing people willingly replacing their functioning brains with a mushed together mix of someone else's art is downright disturbing to me.

[@celltechindustries]

@@LittleJohnnyBrown I understand the sentiment, don't get me wrong. I generate tons of AI art for example, but never share, nor do I ever seek or am interested in someone's else's AI art. Knowing that it's generated at a button press just takes out the personality, I simply don't care for the art if a person didn't make it, after all, I can just do it myself. The art I generate myself is not a replacement either, if I had the money I'd still commission other artists to make the stuff for me. That's not the use I'm advocating though, I'm for responsible and ethical use as a tool to help people do their existing jobs, which is definitely possible. Keen is a great example of that the way they're doing it right now. Using AI not a replacement for artists, but as another tool for them to use. Technology advances, workflows get optimized, that's just the way things go. Most of the things I had to do in my 3d software, or relied on other people for, has been replaced by addons and plugins, non of which contain AI, no one bats an eye at that, it's just when a new trend emerges that people have an issue, because unethical and bad actors are much more in the open, casting a shadow over the people using the technology responsibly and for good.

[@XMaster340]

@@LittleJohnnyBrown You see, here's the difference. You sound like an entitled prick who I would never hire. @Celltechindustries on the other hand seems like a pragmatic guy who gets the work done in time using all the available tools to increase their creative efficiency. So, if you ever get into a position, where no one is willing to hire you, don't blame AI. Blame your child like temper tantrums!

[@aguiasedragoes]

You just released a new update called "Signal" and soon after, a video with the title "Exploring LoRa Models in Game Development." I don't know if you know, but LoRa is a long-range (LoRa) transmission technology. So I believed, for a moment, that we would have some kind of physical accessory, like a mini turret, connected to the game. I think I dreamed too much... lol😅😅😅

[@glabifrons]

I was thinking the same thing! My first thought was "oh wow, interfacing with the real world... we can set up alerts for when our bases are getting attacked while we're away from our computers!" Darn.

[@rodrigosalinas3084]

@@glabifrons I mean, that was already made on Rust, phone notifications when you are being raided using an ingame element to connect with your device xd

[@glabifrons]

@@rodrigosalinas3084 There's a product called LittleBits and a mod that allows you to do the same with Minecraft that my kid was using close to a decade ago. Interfacing with the real world isn't that new. I just thought it was new to Space Engineers, which would have been really cool.

[@MrFaaaaaaaaaaaaaaaaa]

I thought of LoRaWAN too

[@egagnon714]

Wait, is the Voice over AI too?!

[@ondradoksy]

always has been

[@VengeanceMkII]

Folks in the comments did not read the description that the model shown above was trained on in-house Keen art and designs for use in this :P. Any argument regarding generative art and theft is utterly invalid here lol. Still though, you should have included a disclaimer in the vid that the model provided is trained on Keen art.

[@ChosenSquirrel]

Not really true LoRA's do not change the fact your using the Stable diffusion model which depends on that company and what they trained it on. . But personally I don't belive in the kind of copyright people are proclaiming to support , we would need to establish how a that part of the human brain which is creative is different from one of these AI models. We observe reality create an internal model based on the world and our imagination. Would we sue someone for copy right for taking inspiration no . I am against plagiarism however I don't think anyone has a right to own an idea in its entirety. Applied to art you own your art piece but not the inspired art. But also because this was generated with a machine it shouldn't be able to be copyrighted at all.

[@SherrifOfNottingham]

@@ChosenSquirrel the current argument, and a very valid one, is that since generative AI is a tool, just like any other tool, the person using it is given the copyright of the material.

[@LittleJohnnyBrown]

@@SherrifOfNottingham It's not "just a tool like any other". This argument makes no sense. A stylus is a tool. It doesn't replace your thinking or do any decisions for you. Ai does all the decisions for you, but the decisions are garbage and the outcome of this process tastes like dirt.

[@LittleJohnnyBrown]

That doesn't make it better. They are replacing the act of designing a thing with the act of picking the one that is less cringe. This is not art, this is something else and it feels cheap and tasteless

[@Jagent]

It doesn't matter. They are whining because they feel threatened, or they're commies.

[@DrunkenPilotVideos]

As a digital designer I have to say this is slowly becoming a more common practice as it requires less people and time to generate thousands of iterations with AI (that cyberpunk clothes store owner was right). Even in advertisement, next time you look at an advert poster or post in social media there is a non zero chance that the people on them dont exist. They are AI generated.

[@Jagent]

Thank you for sharing the models.

[@chrisgreenwell3404]

Great use of AI technology , thank you for sharing :) Where do we find the DHCAS loras for DL ?

[@chaosfire321]

Definitely interesting to see some GenAI experimentation in game dev, especially when it's used by artists, but did you have to make the AI narrator sound like a skeevy rogue from a fantasy tavern? X)

[@Ecalypse]

Pretty cool way to sketch concepts.

[@aymericrichard6931]

Amazing too share your experiment of generative ia as a studio and thank you for the loras

[@SANDVlCH]

BRB, using this to generate all the content from the DLCs

[@Zer0sLegion]

Trained on SD 1.5, that sucks. Fooocus, my software of choice, only supports SDXL models and LoRas. A shame but I could attempt to retrain the model on XL? I've never done it before but training loRa's is simple enough so I'd imagine there's a way to convert between 1.5 and XL.

[@Chris-cf2kp]

SDXL is overkill on overhead requirement for prototyping and concept designing; not trying to produce high fidelity images. 1.5 has more models available for it too.

[@Zer0sLegion]

@@Chris-cf2kp Yeah, I get that. As I said, Fooocus only supports SDXL and I tend to use that over Stable Diffusion now as I've got it all configured for what I need.

[@raffa0001]

@@Zer0sLegion 1.5 is fine and more accessible, also 1.5 has a lot of perfectly fine models, base 1.5 is fuming garbo, but many out there are really just as good as many XL models in their visual outputs. Also you have way more options and extrensions with comfyui and a1111, from what i remember, fooocus is very limited.

[@kareem2331]

This is really cool, I can imagine how much time and effort this can save. In the modern era where games are taking literal decades to make, this has to be a "game saver", pardon the pun.

[@verence8266]

So many people in the comments not understanding how a generative AI works... Well, I guess in a few years the scare will die out. Nice work Keen, hope it makes things easier and faster.

[@bluestonecreepr]

and their work hopefully will be used for more quality work

[@Angrymuscles]

Sure would be nice if I understood any of this. It doesn't sound like a guide for beginners anyway.

[@Mandalorian_Man]

Keen, you really need a second channel for the AI stuff. It's gonna get confusing mixed up on the SE stuff.

[@LordDrackonel]

Those complaining about the AI art have no clue of how this AI works or its development... But also, about the voice, they dont realize that most people at keen are not english native speakers, just hear Marek speak, he is terrible... thats why they use AI voiceover to make information sound clear...

[@nexttonic6459]

What!

[@nocturne6320]

This comment section shows how badly people don't understand what they're criticising. Cool usecase and to see you're trying new stuff out

[@ondradoksy]

yeah, this literally helps artists come up with new and interesting ideas and people are just screaming that AI is taking jobs from artists

[@james3181]

@@KittenStitches The use of AI defeats the purpose of concept art entirely. People under value concept art just because it isn't seen by the player. This is an important creative process where new ideas for an IP are born. The quality and style of the concept art greatly influences the final product and its aesthetic. Take Fallout 4 for example. In the The Art of Fallout 4 book you can see the concept art and how it grew into the final product. Unlike humans, generative AI doesn't have real world inspiration or the ability to capture the same level of detail and coherent design that routes things in reality and makes them feel believable. All AI art has that uncanny valley effect that makes it feel very synthetic and off putting.

[@ShadowDev6969]

Any form of AI is trash. any company that uses ai becomes trash. AI = the same scam as crypto and nft

[@SherrifOfNottingham]

@@james3181 "All AI art" That's funny, lump everything together not realizing just how much of the world is already coated in AI art that you never realized.

[@FiredAndIced]

@@SherrifOfNottingham Don’t tell them that the content aware feature in Photoshop is also AI-driven.

[@JaegerWML]

Why are you using an AI to read the information?

[@luukvanoijen7082]

Yeah it's really disappointing tbh, a large part of gamedev requires art and for them to be embracing AI like this feels wrong

[@elfferich1212]

Because A.I. is the future!

[@vaughn1804]

You don't have to pay a.i. for overtime 😁

[@NivJake]

Yall realize ai is built into the very core of this Game, right? Good Ai? Don't be to surprised seeing them use ai

[@luukvanoijen7082]

@@elfferich1212 AI could benefit us a lot in the future, but right now it's in a fucking awful position where it only takes away things people enjoy doing. Let AI do the boring tasks, don't take over an artists job. If you don't understand how AI taking over the fun parts of being a human is bad, then you aren't bright enough to understand the benefits of AI either and you should probably shut up about AI

[@claudi_2605]

Update fixed for ps5 no server join ce- fehler please

[@leo.galileo1290]

че

[@raffa0001]

As much as i dislike Keen for being clowns of the game devs scene, after buying space engineers and expecting it to become something more than a barebones sandbox for 9 years, and seeing features now that should have been in the games from day 1, and after seeing solo devs doing better jobs than this, i don't understand the hate for the LoRA approach in game dev. AI is no longer the future, it's the present, if people become obsolete, in this case artists, so be it, becoming luddites and hating on progress, won't stop people from being replaced. You should strive for social, economic and financial change, not for holding back technology.

[@Alex-gc2vo]

wtf are you even talking about? is this generating 2d concept art only or are these fully textured models coming out? why no context or explanation of anything?

[@ondradoksy]

just concept art, stable diffusion only makes 2D images

[@ker6349]

To people who understand the tools being used, this video makes a lot of sense and has all the required context. This video is not a trailer or marketing material for their games, it is a release of an AI model that you can use with a certain type of AI image generator to get images that fit the parts, props, and existing concept art of Space Engineers to allow you to use the same aesthetic, for inspiration or creative purposes. The description has a lot of context and information as well

[@topy706]

3D artists dont need a concept artist anymore. cutting down on cost and time creating new assets

[@HGHsChannel]

LoRa is the death of Game dev.

[@RavlinWasTaken]

GenAi in SpEng before GTA6 damn :/

[@BaliAgha]

annnnnnnd still no story mode.

[@techMan_25]

Yes... definitely something SE needs. Totally.

[@joelmulder]

Cool tech, but we really don’t like the AI voice. It’s cold and impersonal, and makes it feel like whoever is making the video doesn’t care enough to get a human to voice it. I know that isn’t true for KSH, so maybe just have Joel voice these videos in the future?

[@thetoasterisonfire2080]

Here come the anti ai nuts,

[@simonheadless7898]

Keens teaching how to use IA 😭

[@SirBork]

So many of you should be disappointed in the comments, you don’t know anything about what they’re doing. “Good AI” it’s in the name.

[@user-cz6us7ok2j]

Is this why blocks from the Signal DLC look so sloppy or quickly kitbashed from previous blocks? Because the AI told you so? And how does it speed up development exactly, when you're releasing minor updates once a year now, instead of four times a year, and even those are usually just paid DLCs in disguise?

[@reahs4815]

The AI is for concept art only and is trained on already existing blocks so they have the same style. Imagine what you could learn if you only read the description and/or watched the video. With how stupid people in general are and how a lot of people jump to drastic conclusions before they even understand the situation soon I wont have a problem with AI replacing some humans

[@SherrifOfNottingham]

They were using AI for every single DLC they've ever put out.

[@mz00956]

​​@@SherrifOfNottingham so you are saying that they have been using AI to create the DLC way back in 2019? Also where WOULD be the problem with that? You don't have to buy any DLC or even the Game. You can just leave the community if you don't like keen, or if you don't want to support them just download a crack version of SE. I don't get why people are so toxic towards keen.

[@mz00956]

Same thing as above for you @user-cz6us7ok2j

[@ker6349]

@@mz00956 Yeah, this AI is trained ON every part and dlc, but Automatic1111 wasn't even released until years after they started making DLC. That said, a lot of Machine Learning/AI tools have been in use in mainstream technology with fancy names for a while, so I'd bet there was some sort of AI usage, but obviously not in a large scale generative sense like they do here

[@Draxi_1]

The voice is terrible for this kind of video. Also the video by itself is so random

[@alexalcalaortiz368]

wow dude look at these developers hard at work on the next 5€ DLC i mean they put in so much effort per update how could you not love keen

[@nocturne6320]

They still have to create the 3d models, optimize them, create the actual materials and textures. Creating just the concept art is a very good usecase for AI

[@ViniciusNegrao_]

Saves on sketching and iterating through many designs right at the start so they can be more assertive about the design choice so they can allocate 3d artists to produce in-game models which is the most expensive part of the process of making a new block. This way they can make more blocks and with better quality, or focus the efforts somewhere else, you should be happy

[@kahwigulum]

get a creepier ai voice not creepy enough

[@nbmfan]

coming soon: "Exploring AI generated DLC for maximum profit"

[@ondradoksy]

they're just using it to make concept art, it can't generate anything other than 2D images

[@valdemarsimonsen9612]

If we were talking EA/Activision I would agree but stuff like the Signal dlc costs literally around 4 dollars in my country so Keen aren't exactly what I would call greedy🙄

[@SherrifOfNottingham]

They've been using AI since before they released 1.0.

[@AvalineSky]

Generative AI? Nope. I’m out. Will never play or support you again

[@ondradoksy]

it just makes concept art, what are you upset about?

[@valdemarsimonsen9612]

The definition of overreacting🙄

[@davidpodeszwa7010]

@@valdemarsimonsen9612 some people are just so backwards thinking. thank god they are not a majority otherwise we would have never left stone age

[@ChosenSquirrel]

This is the future , you can't stop it, no more then you can stop the sun from setting. In the short term this will actually create more jobs for artists as it allows people to make games more games faster and easier with less monetary cost. If your doing this because of Copyright , people can own their implementation of an idea or an art piece but people should never own an idea in its entirety. This is the bad copyright law that makes it so only EA can sculpt faces .

[@reahs4815]

glad we wont have you here if you react like this to a thing you do not even seen to understand

[@trivalentclan-mizar9591]

I do not like the AI voice. I prefer a heavy accent non-native English speaker.

[@kalthramis1145]

Did yall really use an ai voice for this. What is this, a shitty tik tok?

[@kalthramis1145]

"empowering artists," by using ai to replace them

[@egagnon714]

Disappointed in the use of AI, I loved this game, but seeing you use something that scrapes data from online isn't cool.

[@forgilageord]

It's trained on their own assets.

[@ShadowDev6969]

@@forgilageord This is factually incorrect.

[@SherrifOfNottingham]

@@ShadowDev6969 at the end of the day it doesn't matter, anybody claiming that AI trained on "stolen assets" doesn't understand how AI works, or anything about how artists learn how to make their own art.

[@brunopcem8268]

quite disapointed that you guys are using AI for the game art...

[@SherrifOfNottingham]

quite disappointed you're JUST now finding out. It's not like they didn't have a job opening for "AI Evangelist" like 10 years ago or anything.

[@Wolfie123123]

pay your god-damn artists

[@davidpodeszwa7010]

artists are too slow, this enables them to make concept arts much faster

[@nocturne6320]

The artists are most likely the ones using this, this is just concept art, the actual work of creating the game-ready models is still done by a human. Maybe "pay" attention instead of shouting buzzwords

[@pistonsjem]

artists are gonna use this to make stuff faster lol

[@ker6349]

They do pay their damn artists, this model is trained on the existing blocks and concept art that they were paid to make and this tool will directly assist them with their job LMAO. At least worry about something like the terminator if you need your complaints about AI to be uninformed, unrealistic, and surface level

[@LittleJohnnyBrown]

This is not what you proudly present as your art pipeline. I wish we could have assets that are actually designed. Generated art is cheap af and borderline disrespectful to both audience and art in general.

[@davidpodeszwa7010]

why? when i play the game i dont care that the block was designed by AI as long as it looks cool. i dont see any disrespect to the audience in that

[@ShadowDev6969]

I can no longer support using space engineers or any keen software since they use stolen art through AI.

[@reahs4815]

The model is trained on the art Keen already have in Space engineers and its only for concept art. How is the art stolen if they used a normal AI model? Music covers are not theft but they are basically the same song so why would seeing art (as an AI) and making new art from the old art be theft?

[@SherrifOfNottingham]

That isn't stolen art?

[@FiredAndIced]

Good. Get out. I am also an opponent to generative AI because as it stands right now, the AI tools that are available on the market right now are trained on stolen images from scraping all of the available art images they can find. Training AI on your own models, however, is good in my books. It’s their assets the AI is being trained on. It’s okay, you Luddites can jack off elsewhere. Keen has been advertising for AI evangelist jobs for over 10 years already. If you also don’t like the AI narration, that’s cool, your opinions. At least I know that they are spending resources on Space Engineers 2 solving for fluid dynamics physics as they try and implement water and aerodynamics in to planets. But do please go play Manor Lords instead. I hear that they are _really_ suffering from success.