Realtime Generative Minecraft Down To The Pixel

[@JarrenRocks]

If you'd like to see more explorations into new technology, subscribe for more! A few things I noticed after editing the video: 1. You can see that as I stand still that everything starts to "melt" and lose a sense of cohesion. 2. There's a sheep that appears at 13:45 which is the only Mob I saw in all of my gameplay sessions. 3. At 04:36 I mention switching inventory slots, and it literally would pick different slots than I requested because it was simulating how normally it scrolls around in the real game. 4. This one threw me for a loop by the way: There's actually no "Cursor" in the game, it's just simulating a cursor based on the mouse movements that it receives. 5. 09:25 ASIC means "Application Specific Integrated Circuit" - Hardware that enables better performance than a GPU in this instance. - Thanks @anto_fire8534 for the clarification. 6. I played later and got bubbles by swimming, but then couldn't get rid of the bubbles, so I was just "swimming" through the sky and caves.

[@value1lol]

forgot to pin??

[@JarrenRocks]

@@value1lol good catch - pinned

[@bipbap300]

@@JarrenRocks it is possible to punch mobs, after punching a sheep it often turns into a pig.

[@Geskawary234]

bro like few years will pass and we wont need new games anymore :(

[@KnackiiTM]

I tried switching to the nether by looking at lava pools in the desert but I was unsuccessful. It morphed into other similar blocks instead found in the overworld, I believe this is because there was no nether content given during the model training

[@panw4482]

this is a pretty good simulation of having a dream - weird things are happening, everything constantly changes and it stops right before the best moment

[@Sprlnkles_meow]

I've actually been really interested in this concept because in theory ai is simulated neurons and we see a lot of the same things in ai generated video that happen in dreams where the scene completely changes or objects move, change or disappear all together. Kind of interesting to see.

[@vanskis7618]

I agree, brooo these AI generated content are too similar to dreams....

[@bowser3017]

I will say that even in dreams there is more consistency. You can roughly remember where you were and what you were doing in your dream. But in this AI, everything is constantly changing.

[@randomuser2766]

Glad to know that I wasn't only one thinking this!

[@minlrgo]

@@bowser3017we only need a bigger context window for that

[@Phooenixification]

I see a new hardcore speedrun ahead, beat minecraft in this, just figuring out what to lead it to hallucinate into the endscreen.

[@JarrenRocks]

Yes I actually morphed to the nether in one session

[@neferkar112]

probably getting 9 coal, crafting it into a block and pressing against it so your whole screen is black, or maybe obsidian

[@kill8057]

Black blocks filling your vision while there's any particle falling

[@vpexmc]

So this is what dementia feels like

[@HalfmoonSqueeze]

​@@neferkar112 I tried it, got stuck in a cave with no torches lmao

[@brandonz404]

I got a chamce to play this yesterday. To me, the coolest part about this is that its not a game. Theres no game engine, no real inventory, even your hotbar is "fake". This is a video generator, based on your input, and the last frame rendered. Really cool tech

[@JarrenRocks]

@@brandonz404 spot on.

[@t33mtech59]

Assuning unlimited compute power, this literally extrapolates to the matrix which is crazy

[@mintw4241]

Kinda feel that thats less cool tbh

[@ghighi009]

@@t33mtech59 This literally doesn't

[@baggelissonic]

Not exactly your last frame. The context window is 3 seconds. If it was only based on the last frame it would be an incoherent mess

[@20K_Alan]

Minecraft: schizophrenia edition

[@RhizometricReality]

The lack of self, the lack of memory is emblematic of the enviroment it was birth for, mindless slavery to corperate godcreators.

[@ethanhermsey]

That's perfect!

[@JTCF]

More like playing in your sleep, which this video reminds me of the most.

[@oya4056]

​@@RhizometricRealitywhat

[@nekomimicatears]

​@@RhizometricRealitybrother schizoposting on the schizocraft video

[@Shr1mpinAintEasy]

Non-Euclidean gameplay

[@JarrenRocks]

I think you're being rather hyperbolic

[@nathanmaoah4906]

Both great comments :)

[@Verymusician193]

​@@JarrenRocksI believe its pretty spot on

[@adamPhoebe]

@@Verymusician193 it's a pun!

[@NEKOTHEFNGOD]

anyone seen the non euclidean doom?

[@ozvoid1245]

The best part is the demo ending right before the world either completely breaks or something interesting or scary happens. It just like a real dream

[@shapeshifterspersonalaccou3298]

ayo

[@ghoulishgoober3122]

The clone wars

[@_..-.._..-.._]

It’s just a time limit

[@giacomomeluzzi280]

0:26 I just want to say this is EXACTLY how it works in my lucid dreams. There's a wall, I start picturing a wall, and I can open it and I can suddenly move where I wanted to

[@oshotz]

I would love a datapack that simulates the whole "turn around and you're in a different place" thing, would make for a really interesting playthrough.

[@gwentarinokripperinolkjdsf683]

Maybe a mod that tries to generate the world from an image provided by the ai and let's you actually play minecraft but sorta letting the AI model continue to try and figure it out and use that to further generate new blocks.

[@spiderjerusalem8505]

​@@gwentarinokripperinolkjdsf683, Gwent... I miss Gwent...

[@MarmaLloyd]

You look at the time; look again and it is different but you don't know either time. Just if it is forward or back. You push a light swith bit no matter how hard you push it won't move. The person you were with has changed. The light swith is now a pull chord type but still winter pull. You check the time again and it is later but you can't say when. You read your notepad for advice and you k ow what it says bit can't see the words. You loom between the words and see smaller words written in the sides of the words written. The closer you look you notice a pattern. Like afibanarchi sequence bit with interlocking shapes bit with a distinct work but you can't see the word. .you wake up

[@a1m..]

Would be super complicated if it was made to work any good, constantly replacing/ filling blocks based on real time feedback would probably run absolutely terribly aswell lol, unless you limited it to like 10 blocks under you and only 2 chunks out then maybe

[@oshotz]

@@a1m.. i think a low render distance would make it even better lmao or even if the chunks reset themselves when you looked away, or your inventory randomizes itself when you close out of it, those would probably be more doable

[@Auroral_Anomaly]

Lobotomy Craft v2: Libet’s Delay

[@reclipsel]

Do we have blue search on yt now huh

[@m_r-ock6508]

We do now, its weird ​@@reclipsel

[@reclipsel]

@@m_r-ock6508 It disappeared right in front of my eyes and idrk if I'm schizo or not

[@LarkeyFactorial]

mincarp never b. dame after mr lobtomy 🤑🤑

[@miki890098]

​@@reclipselsame thing happened to me

[@thecwd8919]

I like the cartoon physics. He looker down, and then he fell

[@lockekappa500]

Apparently they used Wile E Coyote to train this AI.

[@wallywalrus17]

This feels like the picture that comes to your brain for a nanoseccond when you think aboutthe word minecraft.

[@ferociousfeind8538]

That's sort of what it is- all video of Minecraft distilled into a disgustingly huge matrix of numbers which, when fed inputs, produces stuff that's kind of like a simulacrum of Minecraft. What you think of when you think "minecraft" is also sort of a distillation of all the pictures and video you've seen of minecraft.

[@jozefcyran2589]

And it does model at a certain level of abstraction resembles how biological neurons work

[@akidevcat]

So they basically just remade Minecraft Bedrock

[@asdf30111]

Still has less phasing through blocks then bedrock.

[@JarrenRocks]

shots fired

[@vindi167]

@@asdf30111 in my playthrough, i remember looking at the sky to teleport somewhere, and then when i looked back at the ground, i couldn't move, and then i jumped because it turned out i was in a block

[@mccraftingtablelmao6204]

11 hour already and no returning fire?

[@roninecostar]

@@mccraftingtablelmao6204 Because its true.

[@AnGerBomB]

Imagine this in vr. You'd probably go insane after a few minutes.

[@Ultracultchaosultimate]

Let’s be honest, it will be quicker

[@calsouth03]

That would be so freaky, I'm already reluctant to play just the regular version lol

[@allayallay-z1z]

and get motion sickness

[@MrDoboz]

it's not 3D, so kinda impossible to make this work in VR

[@Ultracultchaosultimate]

@@MrDoboz I mean just plaster what the game is in vr, you might not be able to have hands with it or something but it will kinda have the same effect

[@rojnx9]

This is basically exactly what my dreams are like! even down to the point of being kicked out right as you find something cool. The internal inconsistency and small context window is exactly my dreams, even the fuzzy graphics and shifting objects, it is literally just my dreams.

[@kadencoates8774]

I'd imagine it's because a sleeping dreaming brain acts in a similar manner to LLMs.

[@salpertia]

Gotta pull up on the gateway tapes

[@Miresgaldir]

I always explain AI to people with the analogy of dreams. It's so uncanny

[@Insert_Creativity_Here]

⁠@@Miresgaldir same it’s so weird Even down to the way you can’t really read/write things in dreams because your brain can’t generate the words correctly, it’s so surreal to go lucid and walk up to a sign and notice that it’s just your subconscious trying to nonsensically jumble a word together.

[@Miresgaldir]

@@Insert_Creativity_Here I love your user name 😂 yes that too! The disorientation. I actually just loaded up the model for myself just then, and the best way of moving around, is just by manipulating the screen to look like a different environment and your instantly teleported - almost like a dream 😊

[@techleontius9161]

4:21 Do you also hate it when you're trying to choose an item and the fabric of reality melts as you speak? 😤

[@Algo_de_Alguem]

happens all time, I hate that

[@prismiemona]

And a pickaxe suddenly materializes in your inventory

[@shadowchasernql]

yeah me too.... you *are* talking about IRL, right?

[@kingofflames738]

LSD Dream Simulator remake looking good.

[@ishipcrimsonandcerluean]

Hell yeah! Glad people thought of that too.

[@3thanguy7]

i saw a comment about this elsewhere and it won't leave my head: i think the reason that it's cropped into a square, rather than the usual aspect ratio, is to cut off all the facecams from the training data

[@bingusbongus9807]

thats actually genius!! that has to be the reason

[@JarrenRocks]

I was thinking a lot about this, and idk actually if they trained on public videos. They needed to capture the keyboard and cursor inputs as part of the training data, so I think it's likely the had people/bots playing MC and captured the data that way. It was TONS of data though, so they were probably all virtual machines. Also square is pretty standard for a lot of these test/basic models. They mentioned with their future hardware it could be served in 4k. Just guessing though.

[@Dr_Pickle]

i wonder if they got permission for said training data. no way is all of this generation going to backfire. sarcasm, btw

[@andybarcia4827]

Nah, the reason is because training these models with square inputs that are power of twos in size is much more compute efficient. It is very common to intentionally crop all training data into a square for computer vision models.

[@SpikyBlade]

​@@andybarcia4827 This is most likely to be the reason. Otherwise, if they trained on data like streamer gameplay, there would be things like donations, etc...

[@AnonymAnonym-s4e]

Does anyone else feel a sense of horror watching this? I think somthing like this could be made into a pretty lit horror vr, trying to run away from *something* but nothing makes sense.

[@curious.biochemist]

JackBond1234 said they'd love to see a horror game like this being sold as gnerative AI when really it's all hard-coded and gets eerie and fuck-y fast. I get the appeal! And I share your sense of dread aswell: It's so foreboding not knowing where you'll end up just after a few blocks, or the thing in front of you shifting and morhping even in scale and distance right as you are staring at it. That was pretty scary! I thought the end was especially creepy, with a horse(??) morphing into a giant blob morhping into nothing but shade... It reminds me of eldritch horror somehow -- now that has some nice potential!! ^^ :D

[@someusername1872]

Fortunately the endgame for Nvidia is neural rendering and they'll probably have consumer GPUs capable of it in maybe the next 10 years or so. Unfortunately they'll probably cost thousands of dollars.

[@katlicks]

@@someusername1872 In 10 years? Unless inflation really goes off the rail, it'll probably be ~500 for a decent one that can run it.

[@NightMourningDove]

Like the backrooms 😂

[@corporatecapitalism]

@@someusername1872always assume super-tech like that is 10 times as far away as they say it is. They’ve been saying “fusion energy in next 10 years” for the fast century

[@jocheeseburger133]

Oh cool so it's like a video game without all the structure and rules and cohesion

[@DumplingDave]

and running it for any amount of time accelerates the effects of climate change!

[@aussieglizzy6998]

@@DumplingDave (still believes in climate change)

[@Moli05]

dude, it's ai. this is incredibly impressive. y'all just hating to hate

[@Moli05]

​@@DumplingDavegive it a couple years and it'll become way more efficient.

[@dotdot5906]

​@@aussieglizzy6998you're probably catholic

[@BraydenTheLoser14]

bro a fully playable ai generated video game when ai media is still kinda fairly new being made this soon was not on my bingo card.

[@JarrenRocks]

Truly surprised myself

[@BraydenTheLoser14]

@@JarrenRocks same

[@Damian-cilr2]

I wouldn't really say fully playable...the items,blocks and stuff in your hotbar just appear and dissapear on a whim,and the inventory and ui really just don't work. Its still kinda playable but not really

[@Cubic_cat]

​@@Damian-cilr2you can play it in real time and without 1 frame per 10 seconds frame rate. This is huge. And resource cost is low, since he is playing it in the browser

[@puppergump4117]

@@Cubic_cat Nearly all the work is done on their servers. The resource cost of running AI models is very high. That is why Nvidia turned into a mutli trillion dollar company overnight, cause they were wise enough to invest in CUDA cores over the years.

[@BuggieBoiOfficial]

When i first saw this I though it was gonna have like 10 millions views or something. You’re very underrated

[@JarrenRocks]

Much appreciated 🙏 stick around, it'll be fun.

[@Ultracultchaosultimate]

It’s been one day

[@joseaugustodasilvagoncal-su5fu]

his content is interesting, and he is a good host (I just subscribed). I think it just needs a more attention grabbing pacing. like, he only said the name of this tool midway throught but I like the way he presents things. it's so relaxing

[@vortexsux]

Minecraft: The Traumatic Brain Injury Update

[@goofyahhrizzler01]

The internal brain bleeding update

[@mayanightstar]

Generative AI logic is so similar to the logic of human dreams it's kind of profound.

[@delta0x]

am i the only one that finds this really cute

[@skeleton819]

@@delta0xlike its funny the way the AI thinks things are correct? yeah i agree

[@delta0x]

@@skeleton819 you do this too but youre not a newly invented form of life so its much less charming

[@BudewFan_]

@@delta0xAI isn’t a newly invented form of life though, it’s just some really fancy equations that usually spit out something statistically likely to look like Minecraft

[@HansZimmer-b1r]

Does anyone else remember "deep dream?" That was one of the first generative AI publications Google made where they ran the ML model for image search backwards to debug it. It produced insanely trippy output.

[@Notpresent-1]

This is low key unsettling ngl

[@genandnic]

The word "video-game" is suddenly x10 more accurate

[@AiAngel]

What’s scary is how close this feels to the progression of human dreaming.

[@DefaultFlame]

Playing it made my brain think it was dreaming. Afterwards I kept having this subconscious expectation that websites should be morphing and changing when I'm not paying too much attention, kept feeling low level surprise that nothing was memlting or morphing, and kept having to remind myself that that is normal. Pretty damned trippy.

[@opperhoofdrampeneer8324]

As someone who has of course, NEVER done LSD, I can confirm this is pretty much the most accurate simulation of what that actually feels like.

[@figloalds]

Minecraft Dementia Simulator edition I think it's kinda lit, ngl I think the solution to this is to inspect the game state and embed it into the model as parameters For example, encode the inventory into a set of "bits" and embed it into the model, in this way there will be a persistent memory of the owned items regardless of the context interval Similarly other information can be embeded too like HP, Hunger bar, XYZ coordinates and world seed (idk if that would make any difference), even if it's embeded as a "bias" that would kinda steer the model one way or another

[@Miraihi]

Right, I've seen pseudo-text RPG games working in SillyTavern. You get the internal logic and variables that are stored in the context and update according to the events.

[@k.k.9378]

Maybe in the future, with a long enough context, you could store raw data for the time of day, nearby mobs, dropped items, biomes, blocks, the code that moves the player... oh wait

[@ferociousfeind8538]

Hi You've invented the concept of "variables" Y'all are so close to reinventing videogame source code that it hurts, and it's burning forests

[@figloalds]

@@ferociousfeind8538 yeah that's true, but its hard do make dementia simulation in deterministic computing And besides, I think it's a fun idea to "merge" models trained in different games and see what happens For example if you merge Minecraft model with a Vintage Story model Or if you merge a Doom with Halo, what kind of crazy "LSD" simulations would result from this? I don't think this works as a product either, but it's an interesting research project

[@BudewFan_]

@@ferociousfeind8538ai bros reinvent stuff that already exists every few weeks but way less efficient, they’ve made trains but bad at least 40 times already

[@Ladenjino]

It feels like a dream after playing MC for hours Edit: bro i just realized that the views aren't past over a million damn Keep it up bro you're so cool 🙏

[@JarrenRocks]

Thanks 🙏

[@lucasboerman3397]

babe wake up. Jarren is playing minecraft again. It was a 8 year drought, but he is back.

[@JarrenRocks]

I was about to start a gaming channel again after playing this.

[@Mertiven]

It's like someone uploaded their dream to the internet and made it into a game

[@UEatPoo]

Watched a couple of these vids so far. Yours was my favorite because youre genuinley excited about it and go into some of the technical stuff, making it interesting to watch

[@JarrenRocks]

@@UEatPoo I really appreciate that, I’m more in the tech space today, but used to do let’s plays, so it was just this weird coincidental, crossover event. I’ll definitely have more game related content, but not sure if I’ll get into a habit of gameplay like this again

[@fendoroid3788]

PhoenixSC needs to see this

[@BierBart12]

He's gonna be devastated that you can't use commands

[@xzydra570]

He already has a videp out where he plays this lol

[@Shmityorshen]

It's fun to think of the emergent gameplay that can happen, the fact that doors can literally teleport you to a new area is basically a new game mechanic, carry doors for extra random teleports

[@9600bauds]

the thing i found the most interesting is that if you are in a dark place and you right-click the ground with your pickaxe in hand, it will place down a torch. it's like it doesn't know what a torch actually is, but it knows that people usually place torches when it's dark.

[@matirr9630]

its like dreaming

[@JarrenRocks]

I like just running around in circles and falling into clouds

[@Foxeyify]

Minecraft inception.

[@dampking]

But less coherent and consistent

[@georgeofhamilton]

This is how AI dreams. We’re all gonna be in trouble when AI wakes up.

[@a1m..]

Nothing like dreaming

[@JackBond1234]

This concept would make for a great horror game. Present it as a generative AI experience, but it's actually manually coded, and have the hallucinations become eerie and seem intentional and dark.

[@No_True_Scotsman]

I was just thinking something like this. Like with doors you go thru and then you go back thru but it leads somewhere else. And like intentional datamoshing.

[@whwhwhhwhhhwhdldkjdsnsjsks6544]

There is no real way you can make something much like this without using AI. Unless every single instance of a hallucination is done manually, but that would be nothing like this given how few there could reasonably be.

[@caut5618]

thats gay

[@No_True_Scotsman]

@@whwhwhhwhhhwhdldkjdsnsjsks6544 You could do this very easily without AI. Just write a shader to give the fuzzy visual look, and have stuff change when you're not looking at it You can still use code to randomise stuff to billions of permutations

[@docgeoffrey877]

@@caut5618 oh, I remember you, you were put into a coma in 2005 and must've just woke up. You're not gonna believe it, but it's been almost 20 years! there was a stock market crash, and then a recession, and then the US elected its first black president, and then everyone thought the world was gonna end, and then a gorilla got shot or something, and then the US elected a washed up reality star as president, and then there was this big pandemic that a lot of people thought was fake, and then- oh, yeah, and using "gay" as an insult - totally lame now. I know, never would've guessed it!

[@runforitman]

I love this I hope it doesnt go away like how the crazy ai art did I loved those images that looked familiar, but you couldn't name anything in them

[@JarrenRocks]

picking a thumbnail was so hard because every frame looks kinda real, but the morphing is the weird part

[@weirdredstone42]

i loved those tooo

[@SealWithoutHat]

This project is way too unique, makes you think about what games will be in the future.

[@SupahGeck]

This is a really interesting concept but I don't think it could ever be more than a gimmick in an actual video game. I agree with others saying it could work in a horror style game, but the incoherence would have to be balanced with monents of consistency to actually sell the vibe. Maybe if a character is walking through an otherwise fully designed game with good mechanics, etc, and the character has a breakdown or nightmare scene that works like this, sure. But even then, a lot of the compelling elements in something like that are story moments like the void of their dream revealing the character's dead sibling or something. Idk, I just feel like once you're increasing the context window to ridiculous amounts and coding in UI overlays whateberver, at a certain point you might as well just make an actual game, with good gameplay feel, story, and level design. This will mostly just give us a distilled/generic version of games we already have with some trippy haze around it.

[@dogonit1936]

no you don’t understand, this is a glimpse into the FUTURE of interactive AI experiences!!! this will totally replace all video games and every game dev will be jobless by 2026!!!🤓🤓🤓

[@sc0pe103]

@@dogonit1936 people unironically think that and that's both hilarious and sad - imagine thinking it would be better to offload creativity onto a machine...................................................

[@harmonicswarm]

@@sc0pe103 you see, if we automate all the creative, interesting, fulfilling and/or fun kinds of work, that will finally give us time to fulfill humanity's true purpose: dying in the lithium mines so the last human can be a Martian quadrillionaire

[@nise6699]

It'd be so cool if there's a way to put a permanent object in there somewhere so multiplayer and monsters would make much more sensible

[@bipbapboop1140]

You articulated my thoughts exactly. This is very cool as a Weird Thing™ you get to check out, but AI generated games have as much artistic value as AI generated images. And for me, it matters a lot if I'm having that "conversation" with a real developer or just with a machine.

[@ExitSignBrokeOnce]

Minecraft with dementia:

[@2DImpersonator]

The weird shifting of the textures is really similar to hallucinations off psilocybin and that's really fascinating

[@TruTrolldier]

I was going to say that about LSD. Done it only once but this freaked me out because i had a very bad trip. It’s scary when you’re disoriented like this.

[@ZorroFox2001]

I tripped after the AI boom and I was like "what the hell why is everything AI generated" while I was coming up

[@Notanothercrayon]

As someone who's planning on trying psilocybin soon, this is an interesting insight

[@gandjalfthegreen2143]

@@TruTrolldier Its astounding how easy it is to forget how disorienting psychedelic experiences are. Playing this pulled up some long forgotten memories.

[@blockyorbits2376]

​@@Notanothercrayon Don't fight the trip and you'll be good.

[@ZonymaUnltd.]

*Context Window* is a _CRITICAL_ term. Crazy stuff. Thanks for this video!

[@YSPACElabs]

I don't think that this is the future of games due to the [totally unpredictable gameplay], but it's still cool research-wise and it may give people insight into how transformers and large models work, manipulate data, and store information.

[@malixcrash4126]

This is.. the single most insane thing I have ever seen in my life

[@hellterminator]

I like the idea of storing the inventory outside of the model. Maybe they could also code something to keep track of blocks outside your FoV. And maybe also some check to make you fall when you're in the air, kinda like a mini physics engine and... oh, wait...

[@JarrenRocks]

You’re right though. The simulation part is more interesting for the times when you want to fill in the gaps, rather than the whole game. I’d like to see what a mix would look like.

[@JaaceB]

at 3:30 you have a bunch of water, but go to a new map to "get some water in here". Maybe you're an AI hallucinating as well?

[@Snow-wv2sf]

Imagine an AI being excited about an AI being able to generate a game for it to play

[@JarrenRocks]

I blame it on recording. It’s harder to use 100% of your brain cells when you have a camera on

[@liquidscreamisnomore]

16:42 close your inventory and turn around, some mysterious evil dark blob appears behind you, you try to investigate it and then the game crashes this is a real life creepypasta

[@JarrenRocks]

I got freaked out pretty fast by that blob

[@Аноним-п1ж]

Damn, I saw something like this in my first "simulation". Something dark was standing 6-10 blocks away from me when I turned around on the last seconds of simulation. But it was probably a morphed darkness in some cave that was pretty far away from me.

[@liquidscreamisnomore]

@@Аноним-п1ж i tried it myself earlier (at 2 am woo what a good idea) and had my inventory opened for an extended period of time, i finally close it and i'm in a monochromatic hellscape that consists of nothing but lines of two colors, with a single small strip of red on the ground i look down at it and i'm now in ACTUAL hell; the nether

[@SioxerNikita]

It doesn't crash. It is timed. Look top right

[@nise6699]

That black blob is a horse

[@iTalickChicken]

7:47 that cliffhanger was closest presentation of dream endings

[@MikeScottAnimation]

So, this is the worst AI generative gaming is going to be and it’s already mind-blowing

[@elijahwallis956]

Bro what are you on this looks like actual garbage. You can’t even play for more than five minutes and everything disappears so you can’t make any progress in the game. Are you actually tweaking rn?

[@JarrenRocks]

Think of this like the Atari days of video games. square moving on screen, pong and all that. Just like that was “bad” visually but impressive technically, this is similar. This is “bad” visually, but impressive technically. And these demos set the stage for things to come.

[@Arnazisti]

​@@elijahwallis956 it's a very cool experience I don't know why you feel the need to put down people who are excited for something

[@madloudnoises]

@@elijahwallis956 12 year old has to have his input because unfortunately there's no laws regarding children leaving stupid nonsensical comments online. Who let this kid have a computer? This is absolutely mind blowing

[@grahamburdick]

curious how it could eventually get applied to the creation of a new game tho if it needs to be trained on an existing one

[@nicolasluiz1538]

This opens space for a completely new game genre. The AI generated game, one of which the gameplay is going to be completely new every time it is played. Almost like procedural generated game worlds like No Man's Sky or Minecraft, but maybe with the entire gameplay entirely different every time.

[@SixDigitOsu]

This was incredibly entertaining to watch, no idea you had only 1k subs

[@fr33kout82]

After playing this for a while I had to let my trust in my sense of object permanence build back up, this is insane

[@bipbapboop1140]

what i think of whenever i see projects like this is how much less fun these projects will become the "better" they get. The entire charm of AI generated stuff for me is seeing them devolve into nonsense (like that ai generated beer ad). When it gets better it just becomes a video like any other.

[@bottlekruiser]

yeah theres a charm to the weirdness i miss with flux that i could get with weird interleaved SD prompts i predict it will be less of an issue with such a directly jnteractive thing though and also when we have weights and code available we can start messing with the inference in fun ways

[@Textra1]

It's like a dream. You're going somewhere to get a thing to bring back to someone, but on your journey to get the thing the world changes around you and you have trouble getting what you were looking for and even if you do, getting back to where you were is impossible. A dream is just a long free association with little to no ability to back trace through the "narrative". This AI stuff is really teaching us a lot about ourselves.

[@masterninjaworrior]

This is a visual representation of what is described in Eldritch Horror. Not the monsters but the contrasting natures of a given thing.

[@corporatecapitalism]

I would describe eldritch horror as being able to know something which you can’t understand but I do still totally see where you’re coming from. Actually half way through writing this I realized seeing something you think you understand then It changing completely could totally be interpreted as that.

[@pichooooo]

The new Minecraft lobotomy update looks great

[@teeterboy4690]

13:43 HOW DID YOU NOT SEE THE SHEEP!

[@JarrenRocks]

I know I only saw it while editing ☹️ if I saw it while playing I would have chased it

[@davidjohn_va]

AI simulator

[@BierBart12]

I love that it instantly transforms into a sand block

[@number9letterk]

He was so determined to keep the village in frame in order to see if mobs were in! :)

[@Cjak001]

This is like that 4D miner game but even more trippy

[@man-from-2058]

This somehow manages to be more bizarre than 4D hyper geometry

[@SandwichGamesHeavy]

That's my dreams, clourised and visualized

[@JarrenRocks]

It's like when you play till 3am, eat a ton of candy, and then try to sleep -> you see this

[@SandwichGamesHeavy]

@JarrenRocks or play until 3am and then do maths until 4 am

[@undefined6512]

Bro dreams in monochrome

[@Hamfrags]

This is incredible! I love it. :D It gives me the same sense of space itself breaking as I had when I first played Portal.

[@bladee512]

Most people think this looks like a dream but to be honest i feel like this looks more like the tv in your head on NMDA antagonists

[@Sailflish]

The gameplay seems like an uncontrolled lucid dream

[@MarmaLloyd]

You look at the time; look again and it is different but you don't know either time. Just if it is forward or back. You push a light swich but no matter how hard you push, it won't move. The person you were with has changed. The light switch is now a pull chord type but still won't pull. You check the time again, and it is later; but you can't say when. Or if it was before you began. You read your notepad for advice and you know what it says, but you can't see the words. You look between the words and see smaller words written in the sides of the words written. The closer you look you notice a pattern. Like the fibanarchi sequence but with interlocking shapesyet with a distinct real 'word', yet you can't see the word. .you wake up

[@anto_fire8534]

9:35 etched is the company they partenered with (I believe). ASIC means application specific integrated circuit. it can allow you better performance than cpus and gpus because its designed specifically to do the specific calculations u want at an hardware level, hence why they could run their model in 4K

[@JarrenRocks]

Yeah I realized when editing that ASIC was the type of hardware in this case. Good to note, I'll update my pinned tab with that info too.

[@Miraihi]

It explains why the performance is so much better than the AI generated Doom gameplay I've seen before.

[@a1m..]

No idea if it’s just me but, A.I incoherence just makes me itch, it’s such a disturbing feeling. Things like this, and just A.I generated videos in general make me feel this weird, crawling sensation

[@JarrenRocks]

I feel that for sure. At the end of the vid there was a big black blob that was disconcerting, I backed up from it at first. It makes me nervous to see what's next sometimes.

[@a1m..]

@@JarrenRocks I used to seriously struggle with paranoia and it just has this hallucinogenic feeling to it lol

[@Lyrog]

This video was both more entertaining and more informative than the videos made by the biggest youtubers, nice work

[@Kiiltec]

Honestly that would be a very interesting game mechanic. Imagine a game that changes in the directions you look. For example you go forward and things are normal, you turn and everything starts to change and barely anything is similar. The twist being walking backwards without looking back keeps the old design of the land loaded. So... aslong as you move straight backwards or forward it stays the same, but once you start turning it all changes.

[@jme7046]

That's kinda what 4D miner is like

[@Kiiltec]

@jme7046 not necessarily. 4D miner is the same all around, you just simply move through 4d space as the game trys to simulate, nothing is technically changing. You just have way more angles of perspective. But visually... maybe a little bit. I'm talking about completely changing the landscape for the most part, with the only 2 things being the same being things you can see directly and things you cannot see whatsoever. Basically moving through space by turning while potentially trying to figure out a path to a point in said space. For example they could turn and find themselfes with the dot being way below them, so they look for something cave like , look at it closely and turn, causing them to suddenly appear in a cave like environment with the dot suddenly being above them or on the same level. It would be quite the brainteaser, hehe.

[@Perithist]

I played this and I got to the nether by looking at something red and subtly moving the camera with it

[@vtechvsmile]

I feel like we're watching the matrix be built around us

[@ddylan4cats]

What could humanity do about it?

[@mareksinister]

No game engine, just ai and you are playing it in a browser. This is actually incredible.

[@novantha1]

This is really promising for future developments in both AI and video games. A major issue with price to performance in video games and hardware has been that the clock speed of processors (GPUs, CPUs) has stopped increasing. Instead, they’re generally going “wider” nowadays, meaning that not all classical software is able to scale to use it. If you think about many game engines, they can’t necessarily use all the cores in a modern CPU to increase the frame rate, for instance. But AI scales remarkably well. In fact, you don’t even need all the processors to be in a single device. You could imagine pre-computing slow moving components in a remote device, and computing fast latency sensitive elements in the device closest to the display, for instance. Imagine not being locked into, say, a single expensive GPU, and being able to combine two cheaper ones from the used market like we used to be able to do with SLI / Crossfire…Or being able to choose to run on CPU, or choosing to run on an NPU, or whatever else have you. The cool thing is that once you get something like this Minecraft demo, it’s actually not that much more technically demanding to do, say, “realistic” graphics (and I mean actually realistic, not video-gamey realistic), or to do any art style you’d like to do. You could also imagine doing more realistic, larger scale fluid simulations, for instance, or crazy physics simulations that are actually realistic…And for essentially “free” without really much more processing.

[@JarrenRocks]

Exactly. If we eventually have a model trained on physics, objects, and interactions, it's possible it could become a completely new type of game engine. Similar to how ChatGPT is like the ultimate text AI, this could be the ultimate spacial AI. Not to mention incorporating this with generative video like Sora or the recent Mochi1 by Genmo.

[@Miraihi]

Diffusion tech is still pretty unoptimized. It'll probably become less resource-intensive in several years.

[@dogonit1936]

i think id rather have a game that runs on an engine than an interactive mp4 file

[@JarrenRocks]

@@dogonit1936 tbh I’m tryinna have both

[@ci.netproductions]

I came to the realization that since you’re able to upload a screenshot to play, you can feed back a frame from what you just played (you can download the video). Meaning, you can feed the last frame of your previous session, and continue what you were just playing. Almost like falling back asleep to continue a dream you liked

[@williamriisager8994]

VR port gonna go crazy

[@SpicyMelonYT]

oh shit for real didn't even think of that lol. But actually probably not possible for a while because there would need to be perfect consistancy between both eye positions. And if there is not it would give that weird effect when you look at two different scenes with both eyes

[@bernatrosello4375]

​@@SpicyMelonYTthat is literally the least of the concerns of what it would take to implement this for VR. Please just stop endorsing the creation of AI slop.

[@SpicyMelonYT]

@@bernatrosello4375 you're a bad person

[@bernatrosello4375]

@@SpicyMelonYT no, I'm a game developer. And I'm tired of seeing the humanity being stripped away from so many aspects of art, programming, writing, hell even games now. I've worked on VR games and know the inner workings behind them, the technology is simply not there yet.

[@SpicyMelonYT]

@@bernatrosello4375 I also make games. And the humanity is not being stripped away because these exist. Its because of the lack of laws keeping human jobs. Humanity is still very much there. We like making art, we like making tech. Humans rock. And whats the point of your last statement? You say it like because its not there yet then whats the point? That same damn argument can be made about EVERY SINGLE CREATION EVER. Hell even a game you work on isn't finished until its finished. But you were already imagining what it would look like finished, what it could do, how it can feel. Its there same thing here dude. And you draw this arbitrary line in the sand because you don't like it. That's bad. You know how many artist hated photoshop way back when? And now those people who use it hate the next this which is AI. Video games are fun, they are creative, they give you experiences, and they make money. None of these thing stop happening with AI nor should they.

[@sugartab]

i had a four hour inescapable lucid nightmare after watching this video before bed... thanks

[@TRFAD]

eventually we're going to have a model that can simulate an entire operating system as if you were on it. With websites and working pages and programs lol could be wild. The direction this stuff is going is really wild.

[@AZaqZaqProduction]

I remember hearing that if you ask ChatGPT to do that it can with reasonable-ish accuracy. You can type shell command and it will simulate them.

[@dogonit1936]

sounds like an amazing idea 😂 i open up a new chrome tab and all the files on my desktop disappear forever

[@montyvr6772]

Why would you want that over an actual operating system that takes exponentially less computation to run and is more accurate? Imagine an OS that just hallucinates files lmao

[@corporatecapitalism]

I really doubt we’re actually going to have that, at least not within our lifetimes. That would be so resource intensive it would just be worthless

[@stevencurtis7157]

I was expecting just video generation, not "we have literally any video game at home." This is nightmare fuel to my game design sensibilities.

[@IgnasiaSundavr]

The giants drink from enders game is gonna be real.

[@quantumblauthor7300]

AYYYY

[@Meminjo]

I cant wait for this technology to advance enough that actual playable games can be made that utilize it fully. One concept I can see these hallucinations being perfect for is a sequel to LSD Dream Simulator.

[@k0v4n0v4]

one of the few appropriate uses of ai in art. simulating a concoction of schizophrenia, alzheimers, and various other psychological conditions.

[@isawadelapradera6490]

this... kinda _is_ art in how absolutely abstract an experience it is.

[@k0v4n0v4]

@@isawadelapradera6490 exactly. its like how dreams feel when not lucid

[@rememberjerry936]

This is an awesome example of how ai work, its a non-sentient pattern module. We actually reconstructed our subconscious how amazing

[@Lemonator32]

What would be crazy is if someone used a camera and turned real-life footage into a game.

[@ColemanRBentz-888]

Imagine a horror game like this where the AI is constantly trying to mess with you

[@budderman3rd]

Wait. Isn't this just how the brain works? Just more connections having a more/less correct version of reality? Like, did y'all know that? The reality you see is a predictive algorithm simulation in the brain, and the brain only uses inputs when it's wrong. It would take the brain an impossible amount of energy to calculate using ALL inputs EVERY SECOND, so the brain evolved to have a more/less simulation of the reality. Like color, for example, their is no such thing as color. It's an evaluationary trait that's only in the brain that we evolved to be able to distinguish stuff around us.

[@ZerosiiniFIN]

Color being a figment is not the same as the brain being a predictive algorithm like the model in this video. Your brain constantly takes in concrete input and definitely doesn't make up as much baseless nonsense as the AI model

[@budderman3rd]

@@ZerosiiniFIN I mean, but like. If the ai had more connections as in the same amount as the brain, would it not be the same? Our brains have like what? Billions of connections or something like that? That's only the simplest explanation of the theory. Being basically a predictive algorithm simulation, or more of a "controlled hallucination". Look up: Why reality is a "Controlled hallucination" Posted by Kyle Hill and then you can probably do more indepth research of such topic

[@ZerosiiniFIN]

@budderman3rd Humans hallucinating is based on actual, concrete, current input. This predictive AI has been trained on terabytes of data and only uses that and the most recent prediction to form the next one. There is no new information other than the player's inout which is not more than a couple of bytes of data. There are some similarities, but I fail to see how your comparison is accurate.

[@budderman3rd]

@@ZerosiiniFIN Did you even try to watch the video dude?

[@budderman3rd]

@@ZerosiiniFIN Our reality is basically only in our brains based on such theory. It's a more complex predictive algorithm, that doesn't even use inputs directly.

[@gajusz4372]

"It's not an acid trip. It's an acid adventure"

[@galvinvoltag]

Babe, wake up! You're dreaming about Minecraft again!!

[@jeremy5602]

I’ve had dreams where I was “playing” Minecraft but I saw everything in first person and felt my body moving with my character in the world, but at the same time sometimes if I moved forward or left or right I could feel my hand pressing buttons on a keyboard even though my hand was carrying something in the game or something (very weird, very hard to describe). This feels like my dreams.

[@jaborbis]

why is the website so pretentious at some points oml

[@JarrenRocks]

It's basically a research paper, so it's being written towards other people in the research field. I just so happen to read those papers and love playing Minecraft 😅

[@Dr_Pickle]

tech bros

[@jeslinmx22]

Caption: “Limited memory over long horizons” Image: mountain. beach. plains. ocean.

[@darkbeetlebot]

This gives me violent vertigo. Like, I got a migraine watching that gameplay. That was horrendous beyond measure.

[@Rallysoldier]

This is a REALLY good visualization of an acid trip. Like, really good. There's something more here; even if it's literally minecraft.

[@IntendedPun]

"Hallucinate" is a very apt word to describe how this looks. Sometimes researchers use it to describe some unrelated stuff (like generative text AI making up facts), but this legitimately feels like how dreams and psychedelic visuals morph. Just walking around a bit takes you to a completely different place. Super surreal stuff!

[@thelelanatorlol3978]

Yes, AI literally hallucinates. When we generate any media, we are directly extracting from the imagination of that AI. Human imagination looks like this too, we just can't tell usually because brain likes to fool itself.

[@BierBart12]

It's not unrelated. Hallucinations are the brain making up stuff, whether it's words or wildly morphing images. It's a perfect word for what the neural networks do

[@Neomadra]

It's a little bit like a high dimension game. When your rotating and everything changes it could be seen as bug, because it hallucinates. Or you could see it as feature, where you actually move in the forth dimension and that's why your perspective is completely changing

[@JarrenRocks]

Imagine if, locked away deep in some generated space, there were a key that would help you progress in the game. Your job would be to hallucinate your way there.

[@jibreel1788]

so when we're dreaming it's just our brain putting things where it think it should be based off of previous input?

[@quantumblauthor7300]

Yeah, because that's its job in the waking world, too

[@luis-sophus-8227]

Not necessarily. If your dreams always look like this then there's something wrong in your brain Dreams can be very coherent at times

[@delta0x]

@@luis-sophus-8227 dude have you read literally any other comment

[@luis-sophus-8227]

@@delta0x Well then that's weird. I know that people with clairvoyance have far more solid and structure dreams than the average. Disordered dreams mean disordered self (not referring to medical term), I think that's what is going on here.

[@delta0x]

@@luis-sophus-8227 everyone has clairvoyance. sometimes i just dont feel like putting a 100% accurate and realistic world in my brain during a time reserved for being away from physicality. its fantasy hours for the little delta

[@zachlap3020]

This is the closest we are getting to living a literal dream

[@TSSmith]

Stuff like this should remain experimental. It's cool don't get me wrong, but this should never be commercialized

[@jacobrutzke691]

How would it be commercialized when its just worst than the original product.

[@cerulity32k]

this will be google stadia in 2034

[@jgofimusic]

Why

[@mr.crowjo_3933]

"Ai is bad womp womp"

[@shyflyf3772]

​@@mr.crowjo_3933yes but unironically

[@arturos.l.2248]

Im glad to see someone actually try to use the mechanics of the game instead of just turning around over and over

[@wydua]

Can you beat minecraft with dementia?

[@O_Obsidious]

I believe the next step of this technology is kind of, incorporating it into the game engine to give it all the context it needs whenever it needs it. Without using context windows for positions and whatnot, you'd likely need to give things tags to tell the AI what everything in a scene is, and with that information it can make some cool stuff, perhaps like Photorealistic Minecraft or even other games, without all the weird hallucinating. Since the AI wouldn't be using its context windows on figuring out what the scene should look like, more could be used to make the scene more consistent with artist vision and previous shots. Interesting stuff, this AI video generation