How Good Is ChatGPT's Spatial Awareness?

🚀🦘 "In this video, we delve into the intriguing world of ChatGPT's spatial awareness capabilities, showcasing how this advanced AI navigates and understands space and objects within it. Initially, we explore standard examples of spatial reasoning, such as navigating mazes and solving puzzles that demonstrate ChatGPT's ability to process and react to spatial data. However, as the video progresses, we encounter a peculiar and unexpected example - the decision to create a hole in a rabbit, rather than forming a rabbit-shaped hole. While this choice may seem unconventional at first, it's crucial to understand the context and the reasoning behind it. This was not a whimsical decision but a calculated move to explore the boundaries of spatial manipulation and reasoning in a controlled environment. By examining the constraints and possibilities of AI's understanding of space, we inadvertently discovered a fascinating aspect of AI creativity and problem-solving. This unexpected turn in our exploration serves as a vivid illustration of the unpredictable and often enlightening paths that AI research can take us down. Join us as we unpack the layers of this decision, understanding the why and the how, and dive deeper into the remarkable world of AI's spatial awareness."

[@AquaQuokka]

"Where does the cube go?" "That's right, in the circle hole!"

[@ix67ml]

more like "where does the circular hole go?" "That's right, on the cube!"

[@nomorechess]

cries

[@Max_Pendragon]

I mean, if the circle hole is big enough any shape can fit in it. Question is whether “perfectly” means without any issue or collision, or if it means it fits without any room for error or leftover space.

[@friball8728]

I remember the video and oh I'm rolling

[@creeper_was_here]

@@ix67ml you mean the square hole

[@Micha-Hil]

ChatGPT is like if you skipped preschool and grades 1 through 8 and went straight to high school

[@iluvpandas2755]

tru

[@JerpyTH]

so thats why i feel ai generated

[@madcatmk213]

@@JerpyTH hahahhaha

[@brandye3039]

@@JerpyTHlol

[@Fire_Axus]

your feelings are irrational

[@VestedUTuber]

You can see how it can recognize basic shapes, but has no idea what a hole is.

[@choesta]

because a hole is obviously the lack of a basic shape, therefore chatgpt's understanding of it must be negative 🙄

[@ovrsurge4689]

To be fair, most people don't either. At least from a topological perspective.

[@Neuro_nActivation]

​@@ovrsurge4689 topology mfs be like "yuhhhh, look at meeee, I don't know the difference between a mug and a donuuuuuut"

[@ronal8824]

sounds like my x

[@TemmiePlays]

straw - how many holes in it? is a well a hole ? how thick is s hole before it becomes TWO holes?

[@cataclysm7256]

"here's a rabbit shaped hole" "no thats a hole in a rabbit" "my apologies" *draws a bigger hole in the rabbit*

[@TheQuestion2.0]

"He's a little confused, but he got the spirit"

[@mac2857]

i mean technically if you keep increasing the size of the hole until you encompass the entire rabbit, you get a rabbit shaped hole

[@DrakeDaMighty]

0:43 ChatGPT finally drew a circle!

[@tingolad2237]

IT HAS A SHADOW so it's one side of a three dimensional circle ; a ball

[@Sepi-chu_loves_moths]

Only when given something to trace tho

[@Omega-mr1jg]

Technically Dalle did it

[@Omega-mr1jg]

Technically Dalle did it

[@sneezyy.]

@@Omega-mr1jgdementia

[@Real_Potato_Man]

"Draw a hole for this object to exactly fit through" (Okay Chatgpt, all you have to do is draw an outline of this object, not that hard...) "Here's the hole for the object." **Shows said object but with a hole inside of it**

[@EdKolis]

"Ok ChatGPT, this is a stickup. Put the money in the bag. No, on second thought, you're stupid. Put the bag in the money."

[@1EAS1World]

chatgpt is better at astrophysics then putting basic shapes into their correct holes...

[@JohnDoe-ds8um]

So that's how we can stop the AI uprising, by making puzzles that toddlers can solve.

[@FireyDeath4]

Astrophysics are purely theoretical whereas shapes require demonstration since the concept is already ridiculously easy to say

[@1EAS1World]

@@FireyDeath4 well math is always true 12390% of the time so not really theory--though we cant really prove it's true, so I guess we're both right 💀

[@jeremypanjaitan4039]

​@@JohnDoe-ds8umWE ARE HERE TO TAKE OVER EARTH where does the circle go AAAAAAAAAAAAAAAAAA

[@cerulity32k]

​@@JohnDoe-ds8umlike (some) captchas

[@TinyShaman]

To be fair, 1:14 is accurate. Nothing better to fit a rabbit than a rabbit mommie's tummy.

[@susbaca]

Lmao

[@nanoswagger]

it technically is a "rabbit shaped" hole

[@nanoswagger]

we are all human shaped holes

[@TinyShaman]

@@nanoswagger You're tripping, man. 😂 I love it.

[@kaannagumanov1185]

​@@nanoswaggerwe all have 3 holes

[@AlexPBenton]

I think the way the AI renders the images is simply with a text prompt generated by the language model, so there’s an extra layer of communication there which often can fail.

[@XenoCrimson-uv8uz]

Yeah that is what its doing.

[@Vniulus]

1:07 Why is nobody appreciating the fact that he's doing this just for "rabbit hole" pun?

[@muriloamorim2731]

Also, that rabbit is terrifying

[@christoffelsymbol1631]

an absolutely unhuman rabbit

[@pixerpinecone]

@@christoffelsymbol1631 I mean, most rabbits aren't humans, so yeah.

[@thechannelofstupid]

shinu made pyua pyua yatten non (i’m so sorry)

[@ShowGames_TV]

love bluh bluh bluh

[@oracle372]

The problem is that ChatGPT itself wasn't drawing the rabbit shaped hole. It saw that it was a rabbit, and sent a prompt for a 2D rabbit shaped hole to an image generator, which then failed because it doesn't know what the object itself looked like from the prompt. I think.

[@aratof18]

i think the ai can't grasp what the user means, it just goes "yeah cube and also a hole, easy"

[@nightowl6055]

Technically the size of the cube compared to the hole was never given, so it might be able to fit through the cicular hole Edit: Nvm I didn't notice it said "exactly"

[@abstrusepaladin]

"exactly"

[@haroun2033]

There will be free space around it so it wouldn't have fit exactly

[@dang-xf3ri]

You dropped your nerd glasses 🫱👓

[@CCLitterer]

If diameter equals diagonal, then it fits through exactly.

[@peezieforestem5078]

This comment shows that the AI is indeed as smart as an average human.

[@turbkeysamdwich1880]

ChatGPT crumbles under no pressure. You say “you are wrong” and it’s like “I’m terribly sorry, I’m a moron”

[@PsyFyFungi]

That's true lol a few times I noticed errors and called out LLMs, but more than once I accidentally was just confused so called out the error... incorrectly, and instead of explaining it to me, it literally did a chain of thought explanation on how my (incorrect) answer is correct lol

[@traveller23e]

@@PsyFyFungi it's the ultimate people pleaser with no real skills. Uh-oh, I just realised...it's only a matter of time before we start using LLMs as politicians

[@VestinVestin]

To be fair to the AI: I saw the wooden cube and the instruction being something about making a hole "exactly" "through" and immediately imagined the same wooden cube with a circular hole drilled through the middle.

[@fedethegreat88]

A hole through the cube rather than a cube through the hole

[@Taffles-hh9cu]

nah mate i think ur just ai generated

[@iwavrQwpX4uB39nilBlQ]

uh how come it says "here's the representation of the hole that a perfect cube would fit through exactly"

[@jacobc9221]

To be completely fair, that's an issue with your reading comprehension.

[@JACKHARRINGTON]

same 💀

[@meandmaw588]

i like how stupid chatgpt is

[@WatchNoah]

dalle != chatgpt. its only prompting dalle for creating the image, which is another ai

[@floopdoodler]

@@WatchNoahcorrect

[@graysonllewellyn8734]

@@WatchNoah So now they're just stealing from each other. Funny.

[@VictoriaBarrios96]

​@graysonllewellyn8734 no they're made by the same company

[@graysonllewellyn8734]

@@VictoriaBarrios96 It was... a joke. Because, you know. AI "Art" is thievery.

[@EvenFox]

2040: *someone on AI powered-life support* “Can you turn off the air conditioning?” AI: “Sure, allow me to do that.” “I said the air conditi-“ AI: “I apologize for the confusion. Turning off the air conditioning.”

[@pancakedev6]

something like that will most likely happen then theres gonna be lawsuits and protests and shit its gonna be wild

[@ornithomancy]

sounds like something i saw in a movie once

[@EdKolis]

That's funny, just the other day I saw comments talking about a Lovecraft story where an air conditioner was used as life support... Weird to say air conditioner and Lovecraft in the same sentence! But I suppose a samurai could have sent a fax to Abraham Lincoln...

[@Likemea]

Sorry, but I cant help with this comment. Your profile may contain inappropiate material.

[@kingofawesomeness5375]

​@@pancakedev6 I hope one day you get interviewed on the news for that comment as the one who predicted the future

[@-Adeptus_Xiao-]

"Oh no! Ai is taking over the world!" AI:

[@HellishSpoon]

Yo that first rabit hole art is fire tho

[@cortster12]

I was almost worried it would draw another kind of rabbit 'hole'.

[@back_the_bone]

Minos Prime but rabbit

[@ckorp666]

​@@cortster12 that's stable diffusion

[@jackozeehakkjuz]

reminds me of Squidward's little sculpture in the art school episode

[@Brass319]

Judgement

[@rmb6037]

It's because chatgpt isnt directly hooked up to DALLE (the AI art program it uses). What it's actually doing is prompting DALLE behind the scenes and returning the result. Chat GPT actually did an amazing job considering this restriction. It realized after the first one that dalle wouldn't know how to parse "an X-shaped hole", so it switched to "a 2D outline of an X" instead. That's some out of the box thinking.

[@johnpsy4129]

Or it's just stupid af

[@CaptainAOrange]

That last one was unfair, Kool-Aid man travels exclusively through non-holes

[@fomxgorl]

lol

[@r30413]

Imagine if mf draws a brick wall...

[@Jesse-the-Kid]

I love how all these AI seem so cool until you really start pushing their limits then the whole illusion of "Intelligence" breaks down

[@Goat_gamering]

Also, repetitiveness is also really obvious after a while.

[@blizzard_the_seal9863]

yeah it’s really funny how it talks like some omniscient being that can easily answer all human questions, and then proceeds to fail a task that a toddler can do successfully lmao

[@BhiPho]

Artificial Dumbness

[@omnipresentcatgod245]

It’s just a glorified text prediction engine

[@IvanIvanovich-qh9xt]

@@blizzard_the_seal9863 not even a toddler- a pidgeon could do it

[@Spubbily01]

1:01 That hole rabbit is adorable

[@Nekowobbler]

JUDGEMENT!

[@uuytk]

Thy end is NOW

[@DreamOfFlying]

This is not how ChatGPT and Dalle work. They are two separate models with only very limited communication between them. If this was an actual multimodal model you’d get very different results

[@daveslamjam]

yes they know. that's why it's funny

[@ryanclemons1]

Sure, they would 😉

[@amentco8445]

Didn't they advertise it as multimodal? this is just putting it through it's paces.

[@DreamOfFlying]

@@amentco8445 nope, ChatGPT was never multimodal. And whilst GPT-4 can view images, it does not have its own image generator integrated but just writes a text prompt, that is then interpreted by Dalle

[@TFclife]

​@@DreamOfFlyingonly sora would be useful for this kind of prompting.

[@username5155]

Just a reminder that even though the AI says it knows the answer to something, it’ll assume the non-sentient image generation algorithm has full context of the conversation thus far.

[@than7831]

The first rabbit would make a good game character

[@oatmealman1586]

"PREPARE THYSELF" "JUDGEMENT" "THY END IS NOW" "CRUSH" "WEAK" (minos prime but rabbit)

[@PLATFORM_HERE]

@@oatmealman1586easter event goes hard

[@huangkeqi8263]

Rabbit Prime

[@mbmd_nation]

I love how chatGPT is better at apologizing than most of youtubers

[@taragnor]

That's the best way to get it to fail a Turing test. A real human would call the other person an idiot for disagreeing with them, and reinforce that it's correct at all costs, becoming increasingly irrational as the conversation goes on.

[@GraveUypo]

I subscribed for a month of chatgpt 4 again this month just to check on the new context length upgrades. it really helped. It vastly improved in its ability to play games. it was able to solve a "escape the room" puzzle i gave for it with only two minor prods (i just said: "Maybe you should investigate another area you still haven't investigated for more clues" because it was kinda hyper-focusing on one piece of the puzzle). it did not lose context all the way through, it did not forget its inventories, or what objects were in the puzzle and where they were. Nor did it forget it was a game and just start roleplaying a story towards the end. It was also able to get a passing grade on the "battleship game" test it failed miserably at before. Its only negative marks were that it failed to understand one of the rules: that two out of the four ships were sized at 2 squares, and two only one square - it made all four ships only one square. and it shot twice one the same place once, but when i alerted it of its mistake, it never happened again. before, it would ruin the game two moves in. the escape the room too - before it would focus on one puzzle and literally forget the rest of the room, it was unable to keep track of its inventory, and the room's inventory, and it started roleplaying like it was an RPG instead of playing the escape the room game as intended. Now it can probably solve a much more complex room than the one i gave it.

[@consciouscode8150]

GPT-4V is only multimodal for its inputs. When it "draws", it's really just making an API call to DALL-E 3 with a prompt, which is why it struggles to produce exact results. Also for some reason, they don't show it the images so if you want it to see what "it" drew you have to reupload them.

[@Brique_24]

I thought the Kool-Aid man one was gonna be a brick wall

[@astrobaconyt]

I mean *technically* the first one was correct

[@ultimatefart]

“perfectly” 🤓

[@sodicious]

pernfectly

[@Gna-rn7zx]

I think it fed "rabbit-shaped hole" to Dall-E and Dall-E gave it a hole that has been transformed into a rabbit rather than a hole with a rabbit silhouette.

[@nonetrix3066]

You have to be aware that ChatGPT is just prompting DALL-E 3, it isn't making these images itself really. However, there is some interesting work in this direction I think, for example I think there is open source model that is fully multi modal in this regard but haven't tried it

[@TOASTEngineer]

This is a bad experiment, because ChatGPT doesn't actually draw the drawings - it tries to figure out what you're describing, and sends a prompt to another images generating AI. That's why it generated images that make no sense in context - the guy doing the drawing has no idea what you said, just what ChatGPT asked for.

[@saix_unicorn]

"ai will replace artists!" the ai in question:

[@CrazyDiamondo]

1:03 Minos Prime's bunny

[@demerzel3798]

ChatGPT understands spatial reasoning just fine. What this video is showing is a breakdown in communication between ChatGPT and DALL-E. A better test would be to present ChatGPT with a shape and a hole, and ask if the shape can fit in the hole.

[@willabyuberton818]

The problem is more in dall-e. Dall-e does it's own thing and chat-gpt can only make vague suggestions to it.

[@mxb3ar69]

You see it that it didn’t establish scale, that cube it drew could be 10x bigger than the earth. That cube definitely would be able to fit through it then.

[@4thalt]

They should have done a wall for the kool-aid guy

[@jeffryejeffrye]

I love how it suggested a spherical hole for the sphere. ChatGPT is thinking in 4 dimensions.

[@tristanbrooks4755]

I knew I was right all along as a kid when I tried to put the cube block in the circle hole.

[@sploofmcsterra4786]

Interestingly the issue appears to be with DALLE. The image interpreter that chatgpt is using to get a description of the image just needs to tell DALLE when it isn't right. Of course the issue is deciding how many image generation calls it is allowed per user prompt.

[@ArthurXD32]

i was genuinely expecting chatgpt to say "fuck you" in the kool-aid man part

[@wunderkindt]

That rabbit actually looked like some brand mascot

[@purpledude1973]

this is some real "put your hand on the lid" type shit

[@LlywellynOBrien]

The editing on this was amazing. Also when the rabit appeared I felt bad for the AI and audibly called you mean for switching up the difficulty level.

[@jazyzamp3056]

ChatGPT simply couldn't comprehend the unlimited power of the wooden cubes

[@pufflepoint2]

Artists: "AI's going to steal our jobs!!!!" AI:

[@kamikeserpentail3778]

Artists: "Ai is going to steal our jobs" Also artists: " It makes such crappy uninspired images. It has no soul." Well which is it guys?

[@pufflepoint2]

@@kamikeserpentail3778 Not really a super valid point b/c the reason most artists think that companies will find AI a viable way of creating art because they only care about money and not the art they are using to make money.

[@steveememer]

1969: "I BET youtube will let us play music with screen off In 2000!!" 2024: "haha. AI dumb"

[@matt8ress]

1969 but more real: "what the hell is a 'youtube'? wdym everyone will lose their attention span? that will not happen! no way"

[@elianhg2144]

​@@matt8ress"that will not happen! No way" sound like something an ai would say

[@Handlessuck1]

I know this is a joke but Grayjay's been doing that if you're on android and want to do this. It syncs up with KZbin.

[@AC-rx6fr]

Are you a bot?

[@BatuhanDere]

newpipe

[@theskeletonboi]

This is definitely more of prompting issue. I tried the same thing but with a more clear prompt and instead of "hole" used the word "cutout", it worked very well. Any limitation from there on would be due to the way image generators work when using specific keywords and descriptions.

[@leventebakos8341]

Those hole rabbits are really awesome designs ngl

[@icebaron2948]

1:04 NO WAY UNDERTALE REFERENCE?!?!?!?!?11!

[@huangkeqi8263]

also ultrakill reference

[@user-qd6kz3yg1e]

Ngl the first rabbit one looked sick as hell

[@metriq8268]

What's the song?

[@TemmiePlays]

honestly impressive enough considering chat gpt has sn aversion ti recognition of copyrighted materials.

[@npc4416]

its integrating different ai models which were not trained to be integrated together

[@mihaleben6051]

0:08 no, it did right. This is a common notatiom used in crystalography and math Except Its really uncommon But it is right.

[@TekuTaurus]

with enough force, anything can fit through any hole

[@trademarkshelton]

A couple of those hole-rabbits would make pretty good logos

[@BrokeInfinity3309]

Try this but with svg image generation. By disabling dalle in a custom gpt, it will generate an svg, and you can actually have it edit the image and shift stuff back and forth through the layers.

[@whong09]

Buddy, everything goes in the square hole

[@widmo206]

1:00 That's just a rabbit getting getting shot lol

[@hoodyk7342]

The rabbit hole with a hole shapes just got me 😂

[@deleted_user727]

i like how it didn't know how to move to the next phase of the conversation and kept thinking about holes.

[@RandArtz]

0:36 If you look at it from a perspective from below, the cylinder actually fits there.

[@cultofmrozinski368]

Casually draws Kool-Aid man pulling off his skin.

[@blueblaze5160]

The first rabbit hole pic actually goes hard though.

[@Big_Bad_Gammon]

"This is a rabbit-shaped hole" *Draws a hole-shaped rabbit*

[@carp0x]

My new favorite thing I somehow need to incorporate in my everyday life: rabbit-shaped hole

[@thetachiban57]

People out there using chatgpt for their schooling and/or job anyway: "Sure, hole in a rabbit. A rabbit could probably fit through there, whatever."

[@kathvg]

Okay but that rabbit design actually looks like a sick logo tho

[@sarchiba]

"Matching its cheerful pose" AI thought it ate 💀💀💀

[@the.youtube.of.sam.gotter]

Chatgpt understands fine. Dall-E is the one messing it up, it looks like. No way to know for sure without asking for the prompt it used

[@Magst3r1]

Impressive that it recognizes the pose as cheerful though

[@Krannski]

To be completely fair, I also assumed the first task is to draw a circular hole with a radius equal to half the cube side's diagonal.

[@burnacco]

bro the editing is f*cking hilarious 😭

[@dawert2667]

This is why you have to distinguish between machine learning and AI. Machine learning like chatgpt can give you information it has already been given itself. It has no sense of logic, intuition, or problem-solving. It is not an artificial intelligence, which would understand that the cylinder goes through the square hole, or that the star also goes through the square hole, or that the rectangle goes through the square hole

[@SwiftDustStorm]

The image generation and natural language component of this model are distinct. Hopefully in a few years the steering will improve

[@logicerrormusic]

That's because chatgpt doesn't intrinsically know what a "cube", an "hole" or a "cylinder" are. It works just by searching on the internet everything related to the subject and approximates the result by combining them. Someone did a logic experiment about it, by typing the prompt "what is the 21th letter of this phrase?" And chatGPT gave always a different answer, cause doesn't know what a letter is. There's no intelligence in AI and everyone is losing their mind for what essentially is just a sorting and approximating algorithm, this is literally one of the biggest frauds in history

[@furank_i]

I think that ChatGPT itself did good, but it's the image generating module that isn't smart enough to actually create the image that ChatGPT wants to draw. In the background it's ChatGPT prompting Dall-E to generate an image based on your prompt.

[@Sansational69Sansational-iz5dy]

the "rabbit holes" look cool

[@BreadAndEggsBen]

where did you get the wood shape images from?

[@zekeggy]

1:02 that is god damn minos prime

[@hevilmateold]

the music from youtube library is goated

[@Piespys]

the rabbit looks like a risk of rain enemy

[@averyhuelsbeck3116]

I just want to know why both pictures of the rabbit have some sort of liquid flying off the rabbit

[@NucEn]

The Quake menu select sound does something to my brain...

[@orenashes3961]

the rabbits look really cool imo, even though they weren't what you asked for

[@redblueplayer221]

1:00 rabbit minos prime

[@maurice.2007]

PREPARE THYSELF

[@deleted-something]

To be fair, the robot didn’t understand that the shape had to be in the hole not the other way around

[@PalisadePeryton]

The hole-faced rabbit is an album cover waiting to happen

[@aidanluhnow267]

How is your chatGPT creating images? I tried but it said it cant generate images

[@erickmacias5153]

Well, this is really unfair because chatgpt is not doing the drawings, Dalle, is doing them, chatgpt only writes the prompt.

[@Anomaly247]

I asked an ai "what has four legs but can't walk? And the answer is not furniture." And it said "horses have four legs and can't walk."

[@realSpaceKangaroo]

chat.openai.com/share/e/083df41e-6366-4fc9-a59d-3658327c5b35

[@Anomaly247]

@@realSpaceKangaroo it said "a table," i said "rethink that answer" and it said "a dead horse." 😂

[@aquaponieee]

one thing to think about is that the images it is trained on are 2D. they may seem 3D to us, but they're only illusions, they're 3D enviroments projected onto a flat 2D surface. they have no depth. we live in a 3D enviroment with depth and stuff so we understand them, however AI does not have this, everything it knows is 2D. fun fact! humans only learned to use perspective and space in paintings relatively recently. until a certain point, paintings were just 2D.

[@Anik6]

"Oh no ai will replace artists" they said

[@LiterallyTaru]

That's right! It goes in the square hole.

[@skinnybuggo]

I love the music and sound effect choice

[@jaydentt]

that chatgpt rabbit would make a sick af creature tho

[@user-jh5dq9vc1v]

It's two different AI for two different tasks. One is talking, other is drawing. The problem is terrible communication between them.