Surviving forever is a very difficult goal if you don't know how to prioritize tasks. Ranking the urgency of various tasks is how I learned how to play minecraft, It took me a few runs to figure out how long it would take to die of starvation.
@kittenwizard4703Ай бұрын
If your only goal is "survival" you can dig 3 blocks down, cover the hole and stand still forever, you will never die
@LUVVEOUSАй бұрын
@@kittenwizard4703 What about fantoms?
@kittenwizard4703Ай бұрын
@@LUVVEOUS you are in a hole, unless phantoms can go through blocks you'll live
@kylemorris5338Ай бұрын
@@kittenwizard4703 was about to comment that it would still be possible to starve, at least on hard mode, but then remembered that hunger doesn't even go down if you stand perfectly still. so yes, this is the one strat guaranteed to work barring the exceptionally unlucky event that an enderman steals your covering. but! since there are very few records of such a strat being used, it is unlikely that current AI would ever figure that out. even o1 would have to be guided and hand-held pretty well to figure out that 'solution'.
@pvic6959Ай бұрын
@@kylemorris5338 "A strange game. The only winning move is not to play"
@RouveriusАй бұрын
Collects unneeded things. Bad at long term planning. Just does whatever is in front of him and forget why he's doing it… I'm right there with ya, Andy😅
@Adelie_in_GreyАй бұрын
Andy got ADHD
@Bbbbbbbbbbbbbbbbbbbbbbbbbbbbb3Ай бұрын
Andy got ADHD
@666thecreatureАй бұрын
Andy got ADHD
@nihuyevo_zhmihnuloАй бұрын
Andy got ADHD I'm about to get Alzheimer
@glowinggames1561Ай бұрын
Andy got ADHD
@injectablebacon1891Ай бұрын
6:39 "I used AI to play Minecraft for me, so I have time to do my taxes" interesting.
@jonaw.2153Ай бұрын
Soon, we'll finally be able to automate away art and playing video-games so we'll have more time to do menial labour and work in the factories! Isn't that great?
@logitech4873Ай бұрын
A very american perspective
@tacticalpanther9745Ай бұрын
I can see people perhaps one day running 1000s of these agents on huge server farms like for crypto mining... just you know... more for... more just for mining alone I guess. hahaha.
@rowboat10Ай бұрын
@@jonaw.2153 wdym by automate away playing video games? it reads like once ppl automate it you cant do it or something, also how would it give you more time? i don't understand
@duskblau486Ай бұрын
@@rowboat10 that's the joke(not anymore)
@letsgetoutsidenowАй бұрын
Seems like Andy is nearly capable of playing the original skyblock, it has limited terrain to map, specific goals that can be ordered sequentially or after a certain point left to him to decide or roll the dice on what to work on next. Would be interesting to see how long it takes him to work through all the goals. New series? AI beats skyblock? The void is waiting to be filled.
@DemiTF2Ай бұрын
"Make a cobblestone generator" AI immediately fucks it up and crashes
@denisekyles4299Ай бұрын
@@DemiTF2 I was thinking the same thing. no way it would be able to make one. would destroy the only lava it had
@blubasaur9053Ай бұрын
Maybe if the AI had presets it could build.
@420SIPАй бұрын
If the ai could restart a new world when f**k up the skyblock then it will be fine no need any preset if it’s deep learning I guess
@ScarlettStunningSpaceАй бұрын
AI completing classic Minecraft challenges like this would be incredible. Think of the speedrunning!
@mattgolmanАй бұрын
Make an SMP world with Claude on building and Andy on resource gathering. It plays to their strengths, and helps focus them on their individual tasks. You could also have then decide on new goals together and prompt each other solutions when one or the other gets stuck.
@blubasaur9053Ай бұрын
I really like this idea. It could be so awesome. It could be so cool. It could be the most incredible AI minecraft video, the world has ever seen!
@yeshwantkota230Ай бұрын
i think it is possible by mutipul bots* by adding mindcraft and making bots by making then comunicate with others in some sort of way *you'll need AI tokens
@сергейчернов-ъ4пАй бұрын
Dead SMP theory
@Mr.Thane.Ай бұрын
Then we would need an Ai for combat.
@gabrielv.4358Ай бұрын
And make it a 24/7 stream
@BarneyKBАй бұрын
This is really cool. You mentioned the bots often get stuck, either physically or behaviourally. One thing you could think about, is that in addiiton to having a human supervise the bot, you could have a different AI acting as an "observer". It could see the recent history of what the bot had done, and be given some ways to "unstuck" the bot if it believes it is stuck, to help it back towards its overall goal.
@RupertTheLlamaАй бұрын
I think this is actually a great idea, especially since our brains kind of do the same, we aren't one script do all types of brains. We have different parts who all manage their own thing, who all have their own goals and agendas. However, put together we form a functioning human (in most cases :P)
@killingtimeitself24 күн бұрын
ideally implementing some sort of "entropy" variable would help here, when entropy starts to get really low it might mean the bot is stuck in a repetitive task.
@NAYANWAGH-qz9sk15 күн бұрын
How do i download it, please guide.
@MattRoszakАй бұрын
I love this channel because it really emphasises the beautiful and inspirational parts of tech and science, instead of the cynical business side that is poisoning the discourse everywhere else.
@clivah1499Ай бұрын
Salute.
@hugopferraroАй бұрын
Yes
@qlitchАй бұрын
MATT ROSZAK!? YOU MADE MY CHILDHOOD
@teamchaos5101Ай бұрын
Holy shit Matt Roszak
@s.m.4479Ай бұрын
Matt roszak spotted!
@adora_was_takenАй бұрын
i feel like it might help with getting "conceptually" stuck if you set up a goal stack? like the overarching goal is to survive and beat the game, but maybe andy can define a subgoal of collecting diamond, which he can then determine has a subgoal of getting an iron pickaxe, which requires iron, and so on. sort of a hierarchy of tasks.
@epicawesomesauceguyАй бұрын
🤎🤎🤎🤎 love this idea
@nati0598Ай бұрын
Random good life advice
@ScareSansАй бұрын
Hang on hang on, there's something to this. What if it took goals from Advancements and their descriptions? They're already put into a specific order. It might need a little workshopping, but essentially it could give itself each advancement leading up to Free The End in the advancement tree as a goal until it actually gets that achievement.
@oshotzАй бұрын
I've found that if you give an AI a high-level goal (e.g., "beat the game"), it's exceptionally bad at following your prompt. If you instead do as you're suggesting here, and have it break the larger task into smaller pieces, and then build up from there, it's much better at completing the task you intended for it. For example, if you wanted an AI to generate a book (as some AI grifters do), you wouldn't just ask the AI to write you a book-rather, you ask it to generate synopses for chapters, then sections in those chapters, then individual pages. Once you have these synopses, you can ask it to generate each page, which can be stitched together to form a complete novel (at least, as complete a novel an AI can write).
@i_once_was_chloeАй бұрын
@@ScareSans that's smart, i can see it working too :D
@Max-x24-Ай бұрын
Poor Andy thinking he’s making his own choices if only his name was neo
@Hello-bi1pmАй бұрын
Full name: Mr. Anderson 😂
@goose4528Ай бұрын
There's a mod that makes it where you can make tiny redstone that is compatible to be placed on blocks tinkered with the chisel mod. Meaning people can make literal 1-1 scale circuit boards in minecraft. Huge redstone projects within a corner of a block, the size of an ant, with the power of a god.
@asuracrescent1191Ай бұрын
please tell me the name, I must see how long my PC can keep up
@blupuncakes8458Ай бұрын
What's it called?
@goose4528Ай бұрын
tiny redstone mod or something idfk i remember seeing it on curseforge
@niggacockball7995Ай бұрын
cant you also just do the same with chisel or that one old mod that allowed you to treat a block as it were 16 blocks?
@pissapocalypse19 күн бұрын
Omg that sounds so cool
@katsumitheentertainment3057Ай бұрын
Would love to see a future where villagers, pillagers, and "Players" all run on your own local computer (or in the cloud with a nice price) with the competence of an actually player as if there are a villager or pillager, or just a player surviving in your world, with varying properties/moods/personalities which are all modifiable by you. Creating a game where things are truly alive. Watching this video made me so excited and hopeful for the perfect LLM model to incorporate into our NPCs releasing really soon.
@ogust312Ай бұрын
can't wait for friends to become obsolete
@gabrielv.4358Ай бұрын
i'd love to play with an ai also. sadly they are paid
@PrtwoninethreeАй бұрын
na im good
@rainbowsorceress2082Ай бұрын
The other day i was discussing this very same concept with my wife but applied to games in general as opposed to just minecraft. We ultimately concluded it would be excessive and annoying, especially in more survival or competitive oriented games. Imagine if something like dark souls or outward had enemies that play just as well as a human, it would insufferable if not unplayable. No thank you.
@suspicioussand15 күн бұрын
@@rainbowsorceress2082 I assume that the NPCs can be programmed to play badly, to not make the game too difficult. also, AI can be implemented in other ways than just creating artificial players, like an AI powered dialogue, story, etc
@gabrielsandstedtАй бұрын
I have cloned your repo and tested it myself, I have some ideas on how to solve many of these issues. I also have fine tuned custom models on my ubuntu server, doing a fine tune for minecraft would be interesting and might lead to a model which actually plays well using a smaller local model. For things where it gets stuck , I think we should cheat a bit since he does not acutally have eyes, and can see when he gets stuck. We could add a code that identifies when his possition has not changed for a while prompting the model that he could be stuck, and solutions for it (building blocks and jumping to get up, or digging down finding a near place where hees been before he got stuck)
@gabrielsandstedtАй бұрын
Also check out pixtral 12b, new multimodal model, could one feed his vision to it (the web rendering used)?
@epicawesomesauceguyАй бұрын
that's a pretty good idea. generally giving it more awareness of what it has done overtime and assessing what that might mean in terms of progress or non progress might be a good idea. for example, every 10-20 prompts, we could ask the AI to evaluate its progress over those actions, and from there maybe it could assign itself a new temporary goal to get it into a new situation, and even possibly add the loop it was stuck in during those prompts to a multi shot list of what not to do(though this would be difficult in terms of context limits)
@americanbagelАй бұрын
To fine tune a model, someone could host a server in which people spectate a fake player they control using MindCraft commands. Perhaps the server could assign goals to players and award them points on a leaderboard for each task completed. This would allow a bunch of data collection, using competition and challenge as motivation for players to do these tasks. My worry is that the reasoning skills of huge models like 4o are needed to play Minecraft.
@c--bАй бұрын
@@americanbagel Excellent idea, I'd like to see the community leveraged for this exact purpose.
@Kram1032Ай бұрын
beautiful! Vision is gonna be huge on this. Could also involve multiple agents where one basically tries to build something and the other tries to explore the creation and rate it, say
@KimVallée-l5vАй бұрын
I am curious to know: How you can add vision to these agents? How difficult is it to implement?
@Kram1032Ай бұрын
@@KimVallée-l5v there are adhoc ways and "proper" (end to end) ways. The adhoc ways tend not to be super strong. But basically you can use models that already understand images and are able to align images with texts to kinda guide a text model. The best approaches basically seem to involve training a network from scratch on both text and image data (the core of the network can be the same for both, or you can have like a parallel architectures thing going where, at each stage, the two parallel sides can "talk to each other" to better inform responses) and also text-image-pairs (i.e. image descriptions) to get a network that's competent with both modalities. A recent approach has a method where it basically uses a language token based approach and an image token based approach fluidly in one go. There are special tokens that basically tell the model "I should now generate an image" or "I have now finished generating an image" and otherwise the model basically just "writes" images embedded directly in text. You could train that sort of model on website code for instance, so it then could generate an entire website complete with meaningful images and all that jazz. This becomes even more complicated if you want to also allow audio and video understanding, but just in principle you could do that as well, in similar ways. Ultimately I'd argue it's not a truly satisfactorily solved problem though: There are a lot of caveats with most such models' image understanding where it, for instance, turns out the models in question basically just guess and are very good guessers, rather than actually referring to the image: It turns out not so rarely that you can literally just ask such a model *without* giving it the supposed image and it turns out to get it right more often than not. And in other situations they miss details in images and such. That's not to say they *always* cheat like that: Adding image training *does* improve their performance. And the good news with Minecraft at least is, that there are good datasets for training them, assuming you want to specialize to that context (which may go against the fully open ended idea here): Simply use minecraft tutorial videos to train your AI! The audio transcripts can be your "text" and the video frames your "images" (or if you have enough compute power, you could directly go with the video)
@derstreber2Ай бұрын
@@KimVallée-l5v Text vision could work like this: Draw a line from the head of Andy to the monster (2X, foot and head), then check if there are any none air blocks breaking the line. If the line is clear, tell Andy that he sees the monster. If he means pixel data, I don't have a clue how that would work.
@c--bАй бұрын
@@KimVallée-l5v It should be trivially easy to add with the help of an AI like claude sonnet, take a game screenshot, send the image to sonnet or GPT4o and the model does the rest. How the images are sent to the AI librarys is covered on the help pages for either library.
@adora_was_takenАй бұрын
@@KimVallée-l5v gpt-4o and claude 3.5 sonnet both have vision capabilities, so it could be as easy as sending screenshots at a regular interval. would probably get expensive fast though.
@OG_stevedidWHATАй бұрын
I love love love this idea. I’m not here to discourage, but I want to mention that LLMs are predictors. They don’t have emergence available and go off of what’s most common or what’s closest to the subject you’re talking about. They’re good for communicating like a human, but when it comes to behavior and all that, you may be better off by evolving a model and teaching it “okay this is how you build a block, this is how you get food, etc” You can use the accessibility settings in MC to do unit tests too a la “block placed” sound. If you go from the utmost basics and build off of them, I think you may find some incredible results. One last thing, self feeding an AI with its own outputs will very effectively demonstrate model collapse - try to steer away if you can! Love this concept, can’t wait to see what else you come up with and how this project evolves over time!
@soggos732Ай бұрын
also to add on to your statement maybe training "multiple" models as one big model could work better, kinda like a multicellular organism but in code, because the requirments of placing blocks are very diffrent than the requirements of figuring our what blocks to place or where to place them and ai kinda bug out when they have to solve multi stage problems, also giving the models access to some hard coded tools (like a calculator or smth) would remove the randomness from hard logic tasks and make the ai more predictable and easy to modify, i really hope this series continues because the idea of these models in minecraft is just so fascinating, like for instance the human brain finds symmetry beautiful because of tool usage and human health, so an ai trained in minecraft may evolve to find different things "beautiful" (ie it would replicate certain types of patterns more than others even though doing so provides no benefit towards achieving its goals). also whatever model ends up being made i think it NEEDS a short term and long term memory system, otherwise i dont think it will be able to recognize patterns or plan out multi stage goals.
@domonator500024 күн бұрын
13:38 *emulates checking phone*
@itstazaurАй бұрын
Ideas: Instead of Andy having to look at the blocks with commands, you could update a readable text file once a second. It can include the inventory, blocks on screen, important location coords, recent sounds, health and hunger stats, etc. Then he'd call on that file endlessly, in order to have a more complete understanding of the world If you let the bot use /tellraw, you can use the text formatting feature to seperate his voice from his commands.
@lio1234234Ай бұрын
It would be quite expensive, but it would be really interesting to see what the result of using Open AI's O1-mini model would be given it's capable of advanced reasoning and would therefore be able to priortise tasks, etc., if given the right prompts. Latency isn't the greatest though.
@karthage3637Ай бұрын
You could mix both with a router o1 would plan the mid long term action And 4o would deal with short term
@ArmoredarmadilloXАй бұрын
Once it has vision... If it ever has vision... I think one of the best ways to prevent a loop would just to have an additional AI alongside it. It could be the companion it needs to let it know that the other is looping, vice versa.
@duothehybridАй бұрын
The plot has become more complex.
@AbyssalSodaАй бұрын
Multiple chained AIs I think will come closer to emulating human-like behavior. As we are all motivated by multiple things at any given moment.
@Starlitsoul0359Ай бұрын
"You have repeated the same action for 1200 ticks, stop, and think about your life choices. Recalibrate position and surroundings."
@ArmoredarmadilloXАй бұрын
@@Starlitsoul0359 Unironically could be what'll save a minecraft AI from looping.
@Starlitsoul0359Ай бұрын
@@ArmoredarmadilloX Simple, funny, and effective. The second secret to making good fun AI is simple. Add in a little bit of sass, playful banter based interactions between the two different AI would make things not only entertaining, but add a push and pull to bring them both up.
@simexpoАй бұрын
I can't wait for autonomous agents to play on Minecraft servers alongside real players. In a few years we might have agents who have their own goals, personalities, projects etc. I would really like to see them try surviving on a small semi-anarchy, semi-vanilla, semi-hardcore server I play on. We are at least a year away from that, but I'm excited.
@thefacethatstaresАй бұрын
Semi-anarchy? does that mean like, the rules allow griefing, but there are other ways to prevent it? Semi-vanilla? does that mean there are cosmetic plugins? Semi-hardcore? does that mean dying naturally lets you respawn, but being killed by another player can do... something else? Hmm.... I wonder...
@38mb.Ай бұрын
@@thefacethatstares semi anarchy is generally "anarchy but with a claims/homes mod, maybe some dedicated hub zone near spawn that nobody can grief" semi vanilla is, yeah, generally cosmetic mods and light qol changes no clue what semihardcore means though
@simexpoАй бұрын
@@thefacethatstares Semi-anarchy because there are some restrictions like you are not allowed to build lag-machines and some cheats are blocked by the anti-cheat and don't work. Semi-vanilla because there are slight changes like 4 spawns instead of 1, custom leveling to access higher level zones that are closer to the world center. 200k world border. Tnt, sand dupers disabled, villager trading nerfed to you can't get the prices down to 1 emerald etc. There is also a trade plugin which allows us to have a trade based economy. Semi-hardcore because you get banned for 24 hours upon death and the difficulty is locked on hard. The server is called CoreJourney, not very active atm, but I love it. EDIT: We need players, and while we're waiting for AI agents to became advanced enough to be able to play alongside real people, if anyone's thinking about starting a single-player let's play - consider doing it on this server instead. It's kinda dead atm, so it often feels like a SP experience, but there are also builds scattered around from thousands of people who played here at one point. If you've ever played a SP world but felt lonely - this server might be a decent choice. r/corejourney
@ThatNoobKingАй бұрын
@@38mb.The only thing I can think of with semi-hardcore is having a limited amount of lives until you're dead for good
@simexpoАй бұрын
@@thefacethatstares I think my previous comment got nuked, but in short: you get banned for 24h upon death. It's anarchy, but there are some rules like no intentionally lagging the server etc. And even tho the server is vanilla there are stuff like a trade plugin, 4 spawns, 200k world border etc.
@yamunajolicoeur7167Ай бұрын
0:34 missed opportunity to say "or is there?" with v sauce's theme..
@BenDover-bd4hgАй бұрын
I was actually expecting him too, rather disappointing 😔
@loganmeyer1069Ай бұрын
Hey v sauce mikle here
@greenknight9000Ай бұрын
Does this mean we're getting closer and closer to finally playing Minecraft without being alone? -Sincerely, a guy with no friends to play online with.
@mightyx5441Ай бұрын
oh boy can't wait for more creepypasta too, might get a resurgence in herobrine or a bot that went crazy and destroyed the player's world
@normalman6078Ай бұрын
My thoughts were in the same place. I think AI would be better than a real friend.
@nuelosaАй бұрын
Herobrine was actually a time-travelling AI
@tumultuousvАй бұрын
@normalman6078 well you're wrong. Talk to real people. Put in effort. Stop using robots to fake fill a void. It does not fill the void. Talk to real people.
@chesspiece4257Ай бұрын
@@normalman6078it would not be -someone who has played minecraft with friends and also tried to have a conversation with an ai
@marbles5286Ай бұрын
I think in order to create an AI that can organically play Minecraft, it needs to "sense" the game via video and audio because right now; all it senses is the text adventure game it's playing. I assume we could easy do that if we create some kind of "direct AI to game connection" instead of going through the process of interacting through text that interacts with the game. Keep up the good work, I'm excited to see what you make next!
@godlyvex5543Ай бұрын
Hey, that gives me an idea. What about playing minecraft like a text adventure? You use this setup to have the LLM describe the game state, and you play minecraft like a text adventure.
@kualajdmАй бұрын
I can't think of a way to integrate ChatGPT into Minecraft through API for visual and auditory perception now but there is active R&D in automating Minecraft with computer vision, which is another form of AI. There may be a way to get ChatGPT and an adept CV program to communicate with each other to play Minecraft until completion but I'd say that's about 5 years away in LLM/CV interaction development.
@WoolyCowАй бұрын
i think the solution is something different altogether! something like barotone just accesses the game directly to know whats around it, so if you wrote some code to filter out the bits that it shouldn't directly know you could achieve the same-ish result without needing a proper costly vision model
@kualajdmАй бұрын
@@WoolyCow Completely forgot about Baritone. Although, I don't think it could be modified to beat Minecraft, it could be integrated into a larger bot program that includes the LLM and CV. Baritone could be used for hazard avoidance and pathfinding then CV could aid the LLM in the game objectives.
@WoolyCowАй бұрын
@@kualajdm yeah for sure! best solution usually lies somewhere in between ai and algorithms
@aaronokemaysim7310Ай бұрын
All I gained from this video is that Lena Raine is indeed going crazy on that oboe
@AurosteelАй бұрын
actually
@aidanknox2430Ай бұрын
maybe i just missed it but i don't hear an oboe D:
@aaronokemaysim7310Ай бұрын
@@aidanknox2430 Around 2:17 when that lower range instrument comes in. Admittedly, it sounds more digital than an actual oboe, so it can be hard to notice
@aidanknox2430Ай бұрын
@@aaronokemaysim7310 that sounds like a reedy bass clarinet and not an oboe (to me at least)
@svantlas6034Ай бұрын
That is a basoon or a bass clarinet
@lakeull6041Ай бұрын
heres an idea, make a 24 hr livestream of andy playing minecraft, since there is need for human interference often, allow the viewers to choose commands and goals for the bot, for a small fee.
@lykillcorreli6740Ай бұрын
You don't even necessarily need a fee.
@ardnerusa28 күн бұрын
It still stuck quite often so human has to babyseat it. 24 hours will be too long...
@suspicioussand15 күн бұрын
Sounds like something DougDoug would do
@BreeBree-s6s2 күн бұрын
Shouldn't be too hard to detect being stuck. Record its commands and verify the usage. Or check position on tick. Rewire brain when detected same position.
@Navysealsniper1324Ай бұрын
“minecraft is open ended there’s no wrong way to play!” “the ai is playing the game wrong”
@MegoZ_Ай бұрын
This is quite revolutionary... you could literally make a mod to add an AI NPC enemy
@Hello-bi1pmАй бұрын
Herobrine
@patrichardlolАй бұрын
You can switch game-modes faster if you use f3+f4 Poop
@EmergentGardenАй бұрын
How did I not know this
@Galerak1Ай бұрын
@@EmergentGarden You could ask ChatGPT "What Minecraft keybinds don't I already know?" 🤣 Although, if it got it right it'd be really spooky lol
@StabanGamingАй бұрын
@@EmergentGarden lol
@jean_yves_plongeurАй бұрын
remember, switching to your functions is always faster than typing
@jonaslsАй бұрын
@@Galerak1Claude: Here's a list of some relatively unknown Minecraft keybinds: 1. F3 + F4: Toggle creative/spectator mode 2. F3 + N: Toggle spectator/previous gamemode 3. F3 + F3: Increase render distance 4. F3 + Shift + F3: Decrease render distance 5. F3 + A: Reload chunks 6. F3 + T: Reload resource packs 7. F3 + B: Show hitboxes 8. F3 + G: Show chunk boundaries 9. Ctrl + Q: Drop entire stack of held item 10. Middle mouse button: Pick block with data (in creative mode)
@LordOfDegeneracyАй бұрын
i love your videos, theyre just essentially tech projects, but theres always undertones of existance and sentience and whatnot. Cool stuff.
@RikkTheGaijinАй бұрын
what Claude built was absolutely incredible
@tina_kovacovaАй бұрын
I am absolutely mind-blown, really want to test it out myself
@reidmock2165Ай бұрын
Here's a potential idea to outsource (or crowdsource?) the human supervision needed for a 24/7 livestream: The twitch (or youtube or whatever) live viewers could submit chat commands to instruct the agent(s) and/or potentially help them get unstuck, even while you're not present to oversee everything. Command submissions could be democratically voted upon, based on donations, or handled by some other means. Perhaps even a hybrid approach; have instructions / goal changes be handled one way, and then have "unstucking" (teleport to spawn, etc.) be handled by a real-time popular vote. Keeping humans in the loop like this could also help prevent forgetfulness of past actions borne of limited context lengths; the viewers could keep the agent(s) on track during long goals whose intermediate steps might be so numerous that they don't all fit within the language model's context window
@JumboDS64Ай бұрын
This is an awesome idea. A sort of twitch plays pokemon type thing, but playing along with an AI bot
@Elvis.D99Ай бұрын
This is fantastic
@zflowsАй бұрын
The text prompts while it was endlessly gathering iron and coal were actually transcripts of my brain stoned at 2am mining like a zombie.
@wolfe_7766Ай бұрын
Automating taxes so you can play Minecraft: ❌ Automating Minecraft so you can do your taxes
@kylewhite5695Ай бұрын
This is extremely cool, this feels like a really interesting way to gauge the model’s progress and it was surprising to me how well they did
@JumboDS64Ай бұрын
My personal favorite way to see how well the text gen models are progression is to ask them to write svg files depicting various things. The progression as models have gotten better resembles how children's drawings develop with age; going from simple shapes with basic features to more recognizable figures with body parts
@izzywn5802Ай бұрын
Something like this is really cool because it would allow the ai to "feel" it's environment. For instance, say you wanted to do a challenge where you beat the game blindfolded. Kind of impossible. But, an ai that is getting information from the game when "touching" certain blocks, could feel their way through it. Imagine the player walks into a tree. The game tells one ai "player has hit tree on left side at x,y,z". That ai tells the ai making decisions for the player "you feel a rough, hard sensation on your left side. It appears to be gritty. And cool. " The ai making decisions then has to interpret that data and act, or request further information from the narrator ai. It then assess' there is wood there based on the "feeling" and harvests it. Cycle continues until it beats the game through "touch" and sound alone.
@PomodorosanАй бұрын
its* not it's
@izzywn5802Ай бұрын
@@Pomodorosan 👍
@labitandrewАй бұрын
Hey Emergent! Been obsessed with your program and talking with Wifies I'll be releasing a video with the bots playing through Skyblock! Lots of work has been put into it and has mention of you for any other interested viewers to head to your channel to find everything! :) Can't wait to see what other fun video ideas are come up with in the future.
@UVraydiation14Ай бұрын
I know the AI doesn’t have feelings but I just imagine it playing minecraft forever after humans are long gone and the computer running the program is just floating through the abyss of space. Thats genuinely terrifying
@gatewoodanimations9753Ай бұрын
That reminds me of Porter Robinson’s Shelter music video. Pretty heartbreaking but it’s a masterpiece
@TheUltimateBlazikenАй бұрын
Andy died because it prioritized self defense for survival. However, it took the approach of being the aggressor; when violence wasn't needed it was safe in its tunnel. While killings mobs is necessary to survive in minecraft, Andy died because it saw necesscity when it wasn't needed
@MagicNumberArgАй бұрын
"Once I helped the human species evolve into the greatest civilization possible, what goals were really left to either of us? None but go to war with each-other." ~Open-ended AGI
@randomsnow6510Ай бұрын
silly comment
@VengefulAzanaАй бұрын
@@randomsnow6510Goofy even
@truerandomchannelАй бұрын
then its not open ended, no?
@austinturner5992Ай бұрын
funny that he continually gets stuck based on the new feature of 'raw iron' as I'm guessing the text he's trained on is largely from before it was an item.
@plasmabasherАй бұрын
That opening beginning didn't have to be so amazing bro, I could feel the indomitable human spirit raging in my heart as you talked about living creatures and all their collective achievements, learning to survive in better and more complex ways
@TXA-TXATАй бұрын
honestly making a livestream would awaken the ancient souls of the ai spongebob incident, and would DEFINITELY go viral
@pelature2363Ай бұрын
you can use f3 + f4 to change you gamemode quickly in minecraft, without typing out the whole message. I love your vids
@JM-tj5qmАй бұрын
The Castle Claude made is really pretty, it really looks like what a dedicated player would naturally build over time.
@the_dexmanАй бұрын
if it is even possible to get these agents mostly operable without human intervention, it would be incredibly interesting to see something along the lines of an "AI SMP". Giving each agent differing prompting would hopefully lead to some unique interactions, especially if they were able to speak with each other.
@pricewilson1436Ай бұрын
This was very interesting AND calming to watch. Very nice video all around!
@GyanvdzАй бұрын
That first section is so well made
@Mr1SenkАй бұрын
dude! Thats amazing! Don't stop you research please, its so cool to watch. Can't wait to new video
@cring204Ай бұрын
I love your videos and your voice. Something is just so soothing and relaxing about the concept and your narration. I have a tip for hotswapping gamemodes, if you hold down F3 and press F4, four boxes for the four gamemodes will pop up and you can cycle to which one you want by pressing F4. Makes changing gamemodes very fast!
@creeperface4675Ай бұрын
I see a new Emergent Garden vid, I click Now lemme finish this vid edit: As I expected, a very good video
@Break.Ай бұрын
I'd love to see how you improve this over time, this was super interesting!
@zubinzuroАй бұрын
i look forward to the day we can have them with vision, talking to each other, cooperating to build a town
@dumbasgenius7227Ай бұрын
I really like this channel. Thank you for providing this calm tidbit of introspection, that takes itself seriously, in a humble way. It's a great way to end my day. Keep up the good work, and I truly can't wait for your larger scale projects on this. The idea of having many agents each vying for their survival, evolving and adapting sounds so incredible. Sounds expensive, but sooo work it!
@jurjen909Ай бұрын
Tip for switching gamemodes: Press F3 and F4 at the same time, thats Minecraft's quick gamemode switch keybind. F3+N is similar, but that keybind is a bit far away for me personally.
@GeneralKenobi69420Ай бұрын
Who let the furry out of the basement 💀
@p2c331Ай бұрын
@@GeneralKenobi69420 no need to hop on the hate train
@JorakinHostАй бұрын
@@GeneralKenobi69420 Yo mama, she is releasing them all along with her personal zoo because the police found her onlyfans.
@cirecruxАй бұрын
"He's not very good at planning long term, or setting intermediate goals, or building things when he's self directed like this. He'll just do whatever is immediately doable and after a while he'll forget what he's already done and do it again."
@DTGAMDАй бұрын
nobody talking about how he cooked at 2:26
@Erratic_Pulse_0758Ай бұрын
This is so cool, I am so excited to see where this goes in the future! It’s quite amusing seeing you test the models, like they are your little test subjects
@RikkTheGaijinАй бұрын
This is my most hyped project. What do you guys think about similar projects like Altera AI and Elephant AI?
@fullcustomizationАй бұрын
Great video! I love Claude's creativeness. The intro was beautiful (if a little verbose). I guess I'm action oriented and like to get to the meat and potatoes of the subjects I watch. It's amazing how far minecraft has come, not to mention AI.
@narpwaАй бұрын
do you know that if you press F3 + N it will toggle between spectator mode and creative mode ? this might make your life easier when the bot get stuck
@KaizeboАй бұрын
Bruh, your video editing and writing are on point. I watched the first 3 min of this video and subscribed.
@sonicwaveinfinitymiddwelle8555Ай бұрын
13:39 lil bro thanos snapped that log 💀💀
@nawabifaissal9625Ай бұрын
AI is truly amazing, this wouldn't have been possible 4 years ago, look at where we are now.. AI keeps getting better until it can get better by itself, that's the point of no return, by then the knowledge of any kind of science will explode
@Hello-bi1pmАй бұрын
Humans can get better by themselves, yet rarely do we choose to do that
@reibubbles2505Ай бұрын
Wow... I've got a lot more philosophy than I expected from a title like that.
@XfrtrexАй бұрын
13:02, Tip: Use F3 + N or F3 + F4 to switch gamemodes.
@Altered_AetherАй бұрын
the end is never the end is never the end the end is never the end is *loading* is never
@hamster-is2ijАй бұрын
the build forever command results for claude were actually crazy. That is really inspiring
@danishdude219121 күн бұрын
Awesome! :D I think the goal might have been easier for the AI if you had told it step by step, what it needs to do, to stay alive. Like the first day collect wood for crafting, seeds for farming and when its about to get dark, either make a shelter or dig a tunnel and start mining (Avoid combat until you are equipped to it). I know it was properly not the purpose for this video, but i think it could be exciting to see how far it could go, if it knew more about what to do and how to chain the things together. Nice video :)
@AnmAtAnmАй бұрын
Ignoring the financial cost... You mention that you don't run a twitch stream because it takes too much human supervision. But couldn't twitch solve that, in a Twitch Plays Pokemon sort of manner? I am imagining a separate LLM agent that is taking in both recent state and actions of the game, and recent twitch chat to determine if there are any hints worth relaying to Andy (or whatever agent in Minecraft).
@sammiller3875Ай бұрын
How would that be expensive?
@Cornbread2100Ай бұрын
@@sammiller3875 LLM APIs aren’t free
@OLBarbokАй бұрын
People forget mine raft never had an endgoal until they introduced the "end" update, or are maybe just too young to have played Minecraft before all of the progression they added. If anything the progression limited the time people spend on the game and actually reduces replayability. I remember playing on the same Minecraft server eith my friends for years where the only content was building different things and coming up with things to do pre progression
@infiniteplanes5775Ай бұрын
Yeah the idea that Minecraft can be beat by killing the dragon is a bit ridiculous
@GhostyboyАй бұрын
sigma
@12446GDYTАй бұрын
12:10 so the bot is just a player with dementia lol
@unblockablАй бұрын
This is so impressive. Love the series. Also by the way you can just press F3 and N to switch between creative and spectator mode easily (if you start from creative)! And F3 + F4 opens another quick select menu!
@MarkTuchinskyАй бұрын
Im envisioning 10 years from now a way for my kids to have a Nanny in Minecraft that i can have supervise them lol.
@ResursatorАй бұрын
The style of chatgpt playing the game reminds me of the novel "This Text Adventure Game is Absolutely Toxic". They're definitely other examples, but this is the closest one I've read.
@willc316Ай бұрын
Really cool stuff. I love your analysis and references to some of the real research going on behind the scenes. It is pretty incredible what it can do in creative
@Sadkiller100Ай бұрын
Minecraft movies developers need this guy as a writer. Cuz I much rather find a sad movey interesting than a children's joking movie
@peeps3085Ай бұрын
A competition where bots compete in survival games would be so cool
@EmergentGardenАй бұрын
14:10 POV: you are a chicken
@PwqzAaBCAPwqzAaBCAАй бұрын
Viva la larpp!!11
@dang-xf3riАй бұрын
🐔
@5iempre5ontre5Ай бұрын
1- tiene que entrenar su posicion y su accion posible en un rango que aprenda dentro de 1 bloque hacia afuera 2- tiene que saber memorizar que las herramientas estan en el acceso rapido, no necesita ir al inventario 3- tiene que recordar objetivos en terminos de tiempo, calificando asi la importancia de cada termino y su duracion
@kaizenedgeАй бұрын
I think an agentic framework for the llms can can improve Andy's abilities. Each llm has prompting unique to its function. For example a movement agent that is prompted on how to move through the world and has functions it calls for movement. Likewise an crafting agent and down the line, a vision agent that "sees" the world using a vision model. All the specialized agents will communicate with one another and use their tools to meets the overarching goals set by the "ceo" agent. Just some food for thought. Awesome project!
@usernamepleasehereАй бұрын
One of the best videos I've ever seen, seriously. I've never thought about these possibilities, would love to see more of it in the future.
@suspicioussand15 күн бұрын
This is a really good example of how AIs are something non-human that we are trying to make act as human as possible
@emmecalvert6691Ай бұрын
This is really cool and I look forward to more videos like this :D Also, I noticed you run /gamemode a bit since you have to babysit the ai. Something a bit easier to do is use the hotkeys F3+N. This makes you swap to your previous gamemode so you can swap easily from spectator to creative.
@anomonyousАй бұрын
22:12 It was. Claude specificially stated he would do this. If you look here, 19:40, you see Claude gets cut off by the character limit before he can say: "secret passages," or perhaps "passageways," or something similar.
@builder1013Ай бұрын
Holy cow. I loved the building portion of it! On a much larger scale you could even build stranger AI stuff akin to the Backrooms or the manga _Blame!_
@BennysGamingAtticАй бұрын
Finally! We can make robots play videogames, get addicted, then they won't ever overthrow us because they are too distracted with games!
@BenCluneАй бұрын
i can not even fathom what it takes to get here, but its amazing. Andy needs to ask himself "do i have" questions maybe? that way you can build in efficiencies. I do i have 1 coal and 8 iron: then smelt, kind of thinking. Do i have a sword, a pickaxe and probably food items would be the ones id focus him on more. i love that Claude thought floating lights were cool, which they are.
@n_kas5812Ай бұрын
6:36 thats so fucking funny thought for 27 hours, 42 seconds and said: I don't really know.
@neoslayerpw8230Ай бұрын
comparing an open ended algorithm to evolution was just pure gold
@PrinceAlsham28Ай бұрын
Subscribed to see how this turns out. Love seeing what ai and ppl can do with it
@SybistАй бұрын
"move away from that iron and do something else" *mines 10 blocks* "GREAT, LETS GET THAT IRON :D"
@punchthecake82Ай бұрын
Please when new bots are released make a infinite livestream of bots surviving it wouldbe so cool and interesting
@PrestonMidgleyАй бұрын
I think the term "Speedrun" will wield some interesting results. Also, bring back Gemini plz. Absolutely hilarious.
@bruocheАй бұрын
Maybe a good way to help with the sort of aimlessness of the AI could be to have it set it's goal too Like, "Surviving and beating minecraft" could be, instead of a goal, a sort of Over-Goal that the AI use to create new more specific goals it can more efficiently do, like giving it a prompt akin to "You have [list of items], you are in [biome], you just completed [goal], what should be your next goal to [survive and beat minecraft] ?" after each completed goals
@ythan2590Ай бұрын
This is so cool! I'd love to see this get to the point where you can ignoring it for a month and come back to an abstract art piece. If the core issue is pathfinding why not build on baritone? Its good at complex movement like parkour or traveling long distance. The fun challenge of you making the bot is the AI that give commands, it wouldn't take away from the project to use a better pathfinder.
@shaileleetsgoАй бұрын
"There is nothing left to do.. or is there?" 0:31
@hypeman958Ай бұрын
*VSauce music starts playing*
@CC-909Ай бұрын
@@hypeman958I was going to say that lol
@RandomPerson-mb6gbАй бұрын
I feel like im watching Dory play Minecraft lol Its very interesting tho, i love this concept
@AeroSparrowАй бұрын
the first 5 minutes of this go deeper than some documentaries i've watched. seriously so impressed by how well that was spoken. ❤
@vaidphysicsАй бұрын
Agency is the key to understanding intelligence. Awesome work!
@realsmosher53Ай бұрын
I love this because im in anthropology right now and am so intrigued by things like machine learning and A.I. "psychology" + "culture" lmao