DALL-E 2 is… meh

[@NN-kd6wg]

Great video! I think you should at least put an asterisk after "meh" in the title. Sometimes the contribution of a paper is in doing all the leg work to set a new bar - i.e. prove some level of performance is possible if you get all the details right. I think a certain level of clickbait is necessary to survive on KZbin, but it shouldn't be taken so far as to be semi-hostile/dismissive to the researchers who are in the trenches trying to get this stuff to work "in the real world" rather than just in theory.

[@TaliwhakerRotmg]

Agreed. Very well put.

[@MixMeMcGee]

Yes. Very well said!

[@wiredvibe1678]

Can't clickbait with nuance

[@alexyz9430]

@@wiredvibe1678 idk man, Tom Scott likes to put explanations in his description / pinned comment to acknowledge context which might be missing from his video titles. so yeah, you totally can have clickbait with nuance

[@wiredvibe1678]

@@alexyz9430 you just have to disagree 🙄

[@CeoLogJM]

The fact that there is little new here and yet we still perceive it to have leaped to a commercial level of quality might be a sneak peak to how far older research will be able to take us just by further application.

[@EdanMeyer]

Older research definitely holds a lot of unexplored gems! For diffusion models I would say the combination of diffusion stuff from graph theory with newer deep learning techniques is a new direction (afaik), but that was not done in this paper

[@jichaelmorgan3796]

My thoughts were more, uh, if this is the ai we know about,...

[@michaelsamoha7342]

אתה מפתח ai?

[@StefanReich]

I'm really surprised how you arrive at the verdict "meh", lol. *DALL-E 2 is the most impressive image AI ever made.* But thanks for the architecture deep dives.

[@spiderspyy]

Pretty good click bait I suppose

[@liangyuaq-qoyunlu407]

you only see a few choosen samples so you dont know how it works if you give it unusual inputs

[@ivoryas1696]

Stefan Reich I would think it was a little more GPT-3 than simply DALL-E 2 being the most impressive, but agreed.

[@munchkintaker4263]

@@liangyuaq-qoyunlu407 Many people have already tried to give abstract, not very meaningful, prompts The results are still pretty good.

[@StefanReich]

@@ivoryas1696 I was actually underwhelmed by GPT-3, because everywhere in GPT-3's output you can sense that it doesn't ACTUALLY understand what it is saying. It constantly makes consistency errors. DALL-E 2 seems much much better in that respect

[@jaredf6205]

It’s interesting how the errors and issues with this seem to resemble the same kinds of issues a human imagination would run into. Almost like they are emergent from a similar kind of optimized fundamental method of understanding the world that both evolution came across and AI researchers. I’m excited to see how much we can learn about ourselves in our study of AI.

[@julian78W]

Fun fact, CLIP is the architecture that correlates the most with neural activation in the visual cortex.

[@wiredvibe1678]

I think this is pretty similar in function to whatever component in our brains generates mental images from memories. Change my mind

[@Im.Smaher]

@@wiredvibe1678 No. You’re probably right

[@leoisikdogan]

6:02 Yes, I was wondering the same thing! They could have trained it directly on text embeddings. The fact that they needed this prior shows that CLIP's image and text embeddings are not as close to each other as I imagined.

[@drorsimon]

I think that the main reason for using the prior is that it increases the variability of the results. I base that on the fact that the GLIDE model didn't have such a block in it and it generated very realistic images, but, not as diverse as DALLE 2.

[@billykotsos4642]

Meh? This would be considered magic 10 years ago !

[@badplayer4348]

It's still considered magic today

[@MikkoRantalainen]

Great video and I think including that rant was good thing. I think the "meh" part was the only thing I'd change but only because I think saying something like "I would have preferred them being more open about the actual architecture and technical implementation" would be more accurate than "meh".

[@Mark-Wilson]

wdym meh its spectauclar its not supposed to be magic its getting better and better

[@EdanMeyer]

I should’ve clarified this in the video, but I specifically mean the research contribution is not that great. I didn’t learn much of anything new from reading this paper because it’s not doing much new as far as I’m aware. That being said the results are still spectacular. I don’t mean to diminish the results because they are very neat, just to say that there isn’t much to the paper itself besides that.

[@Mark-Wilson]

@@EdanMeyer oh alright. I understand.

[@Dannnneh]

@@EdanMeyer You should change the title, DALL-E 2 is anything but "meh".

[@Madison__]

@@EdanMeyer Please change the title then haha. It's almost misleading

[@johnnieblair2325]

@@EdanMeyer Then maybe your title should be "DALL-E 2 is... spectacular" lol

[@hypersonicmonkeybrains3418]

Does anyone find it strange that in dreams you cant quite read text properly, just the same as this model cant display text properly?

[@FalkoJoseph]

Probably unrelated, but still… that is deep

[@Rctdcttecededtef]

What are you saying exactly

[@dajair01]

@@Rctdcttecededtef that in dreams you can't even imagine text and this AI thing also can't do it. Spoopey

[@Rctdcttecededtef]

@@dajair01 why that is spoopey?

[@meghanroxlots]

There is a relation here, yeah. The point is that it turns out language and images are extremely different, so the human brain has separate systems to handle them. The language parts are less active when we sleep, giving the weird dream text. This relates to DALL-E because it was trained on images, and since text is quite unrelated, it only tries to match the visual patterns of words, not their meaning. So yeah, you can think of it like a person with their visual centers on and their language centers off... which is pretty similar to a dream!

[@axcolleen]

I agree with other people. This is still hardwork and the clickbait can be misleading. Other than that you explained it very well!

[@LarsRichterMedia]

One question I've asked many times and never received any official answer to is: What photos have they used to train the model? I heard someone say they scanned through Instagram among other internet platforms. If that's true, the next question that comes up is: Have they potentially used the works of photographers, illustrators, concept artists, designers etc. without asking them? When a human being surfs the internet looking at basically thumbnails of copyrighted original works he can use those thumbnails to get inspired and create his own original artwork as long as it is different enough from what got him inspired in the first place. An AI, like Dall-E 2, could do the same just obviously on a superhuman level. This wouldn't feel right at all, especially if OpenAI is going to be making a profit while all those artists whose works got automatically harvested at lightspeed receive nothing but a potential kick in the butt from their clients. I really wish OpenAI would shed some light on this. If I've overlooked where they talk about it, please someone let me know about it.

[@be1tube]

Worse, there is a paper (conference talk?) from Google showing that you can recover the original images from a trained model unless there were a lot of similar images. So, open AI's model actually contains a copy of each of its training images - potentially setting the stage for later copyright action.

[@LarsRichterMedia]

@@be1tube hey Eric, do you have a link for me? Would love to read/listen to that paper/talk

[@EdanMeyer]

They have not given an answer to this or released the training set. Why they haven’t done so is something I’ll leave for you to form your own opinion on. On top of ethics it also sucks because to properly evaluate a research contribution evaluating the dataset is also important

[@johnvonhorn2942]

@@EdanMeyer The irony is they want to marshal AI in order to prevent harm to humanity or have the AI used for nefarious purposes.

[@NotASpyReally]

No other video explained this as well as you did. Thanks! It's really interesting to understand how it works and I definitely understand better after watching this. I don't think it's "meh", though. It's still amazing even if you get how it works.

[@xefoam]

Hey! I think I can break down the need for a prior. Multiple images can have the same captions, because captions dont have the same level of detail as a full image, ex: "some guy" could be ANY guy! so we need some way to generate more detail from the caption, whats the guy wearing, whats the guy doing, etc. So the prior does exactly that, it just adds more detail to the embedding which allows it to create a more coherent image that is based on a visual style rather than only the semantics. I hope this helps!

[@xefoam]

this is worded a bit weird? because you can argue the visual style is technically semantics! but the thing is the captions in the training data are just not detailed enough to describe every single thing in the image, which the prior aims to do.

[@xefoam]

this prior also helps it create more visually distinct images, since the prior does not generate the same output for one input every time

[@alverdenstop1013]

why the clickbait ?

[@DistortedV12]

Can you do one on Eric Jang's recent blogpost about LLM startups and how everyone is leaving Google to do AGI work privately

[@collinmc90]

Have you seen the things Alan Resnick has been encouraging Dalle-2 to make? Its truly bizare art.

[@craigdickson9345]

Thanks so much for breaking this AI down into an easier to absorb manner. It's beyond my level of comprehension in this moment, so bookmarking for my future self. I hope someday my future self will come back and reply here with proven knowledge of understanding this shit because you set the wheels in motion :)

[@EdanMeyer]

Keep at it, it’s worth the effort! This is an incredibly interesting field!

[@therainman7777]

Just a heads up, “autoregressive” does not mean predicting its own inputs. It means predicting what will come next based on its own previous output.

[@TheRainHarvester]

I want to add errors into the image tokens (corrupt a few bits) and see what gets rendered!

[@superjaykramer]

how do you get access to Dall-e 2??

[@NappyWayz]

I think DALLE is pretty cool. The problem is I can see people using this to make NFT art. This looks so much better than most of the NFT “art” out there and it seems like it took just as much effort from the artists.

[@FunctionStore]

Thankfully it will be available to everyone so if everyone can make NFTs it will lead to further dilution of the market (and eventually the fall [hopefully])

[@ciarrawansley8511]

You forgot one more thing. You know some people will use it for +18 art right?

[@FunctionStore]

@@ciarrawansley8511 according to the creators they filter NSFW keywords/content

[@sebipayne]

You can’t use DALL-E to sell the images generated with it. It’s in their terms and conditions.

[@ciarrawansley8511]

@@FunctionStore good!

[@drorsimon]

Thank you! I think that the "prior" is one of the main properties that discriminate this model from GLIDE, isn't it? Of course in addition to using the CLIP text embeddings instead of some other encoder which I think they used in GLIDE. I believe that the prior actually increases the diversity of the produced images.

[@cedricvillani8502]

Hey someone is actually reading THE paper, 🎉congratulations🎉 I think you’re the only one on KZbin that’s bothering to do so, (I haven’t on this one yet😂) if more people had they would see how NOT magical GPT-3 is.

[@EdanMeyer]

Glad you enjoyed it! There are some other great channels that do paper reviews but it does look like most of them skipped over the paper on this one.

[@diogeneslantern18]

I'm upskilling for AI and Blockchain to use in a corporate environment and it's as clear as day when I peruse blockchain forums that 90% of crypto bros don't know what's going on under the hood lol. I'd bet most haven't even read the various white papers.

[@CommanderCraft98]

I am not sure about your criticism at around 7:30 that the model is trained 95% of the time in the conditional setting. This number is 90% for the big model which they train in their other experiments and also was 80% in GLIDE. Therefore, I do not think, the experiment is unwarranted. Maybe the results are slightly too optimistic, nevertheless meaningful. Please correct me if I am wrong.

[@aiandblockchain]

Very interesting and detailed, thank you!

[@Alexander_Sannikov]

what if you get an image, get an RL model to caption the image, then use another RL model to generate image from the caption, then caption the result, then generate an image from the new caption, etc. and then do this process in a loop, say, a 100 times and see where it ends up. this is also a type of experiment that can produce more interesting results when the programs involved are good but not particularly great, otherwise they'd just converge to a stationary point quickly

[@zohairhadi8087]

hey edan, I got access to DALL-E2 a couple of days back. Would love to talk to you more about it.

[@EdanMeyer]

For everyone saying this is clickbait, I wanted to expand a bit in regards to the title, "DALL-E 2 is... meh": I'll start by acknowledging, yes, there is obviously a level of clickbaitiness going on here, but a good portion of my thoughts on DALL-E 2 are genuinely that the research presented in the paper, which is the focus of this video, is indeed average. Not great - not bad - it's just an okay paper (which is more than what I can say for many papers to be honest). While the results of DALL-E 2 are quite incredible, the "meh" is not meant to refer to the results but the quality of the research. It is clearly my bad not making this clear earlier in the video (I only talk about it at 13:27) which is something I will work to improve in my future videos. That being said, there are a couple reasons I think the research in the paper is average: - DALL-E 2 uses a mixture of diffusion models and CLIP, neither of which is anything new. They make some small changes, but besides the scale it is no far off from what's been done before. - The paper provides very little insights that could be useful in future research (aside from where they talk about guidance, which is relatively minor). - The justification they use for their choice of architecture (generating priors as an extra step specifically) is very biased (more details at 6:03), and giving bad justification is worse than no justification at all imo. To reiterate, the research paper on DALL-E 2 is not bad, but it's far from astounding. You are welcome to disagree, but I wanted to make this post to clarify why I chose the title. Also thanks to everyone who has provided feedback! I'll be sure to improve on clarity in the future.

[@weighttan3675]

nice try 👌

[@alexyz9430]

nice try 👌

[@MochaRitz]

You acknowledge its clickbait and refuse to change the title because of that lol, you only clarify to those who clicked

[@countofst.germain6417]

I haven't used Dall-E but from what I've seen it's not much better than Midjourney. I think it's incredible but it's over hype. Do you know if DALL-E has weights like Midjourney? With Midjourney you can assign weights to your descriptions. For example the colour blue as negative or positive numbers to tell it how much of a certain thing you want in your pictures or a negative number would say avoid it as much as possible.

[@vgaggia]

I wonder if we'll get dedicated AI acceleration in the future almost like how gpu's became a thing.

[@Jianju69]

These images are not expensive to generate at all. Any personal computer could generate these images, albeit limited memory and crunch would impose longer generation times.

[@AI_ALVAREZ_INTELLIGENCE_AI]

I got ideas what if use two deferent image create unique artwork without similarity.

[@chickenp7038]

yesss this is what i’ve been waiting for. trying to reproduce this

[@EdanMeyer]

Best of luck with the reproduction effort!

[@bardobaldor7273]

clickbait "meh"

[@omeadpooladzandi9786]

Lilian's blog is also nice for diffusion models

[@Knightimex]

OH MY GOSH, This is crazy, quality level I would actually buy, This is GReaT! Also: DALL-E 2 is… meh error

[@moslehmahamud]

really nice walkthrough!

[@Colorado-Coyote]

Thank you for expanding my brain

[@MayorMcC666]

nice clickbait

[@RalseiGaming]

i wouldn’t be surprised if ai video will be possible in the next 5 year and a system were you suggest a script and the ai will write a script generate voice and incorporate it into video

[@Yrenne]

This is possibly the greatest invention of the decade and you're calling it "meh"? Or is this just a poor attempt at clickbaiting? I suggest you make a better model if DALL-E 2 is so "meh" to you.

[@shadowkiller0071]

You're missing the point of the video. The focus is on the paper. He's saying they took research others had done and simply scaled it (i.e. bigger neural networks with already existing algorithm), so the actual research contribution is not that big. It's not about the overall contribution. > I suggest you make a better model if DALL-E 2 is so "meh" to you Keep in mind it is mainly a money issue, not a skill issue. If someone gave me money to buy a lot of GPUs I could easily make and train DALL-E 2. The algorithms already exist and are open source, but training them at the same level as DALL-E 2 requires lots of expensive graphics cards.

[@Yrenne]

@@shadowkiller0071 No I'm not. He's using a contrarian standpoint to get people to click his video, while the video itself just talks about how the AI works and actually makes a very good point for why it's NOT "meh".

[@axcolleen]

@@shadowkiller0071 they didn't say it's meh, but the opposite. Maybe saying DALL-E is not innovative is a better clickbait.

[@panta_rhei.26]

@@Yrenne Using an attention-grabbing title on KZbin so people click on your video? This is an outrage!

[@cortster12]

@@panta_rhei.26 It is when it's lying.

[@godbennett]

Excellent video

[@shgidi]

Love your videos, but the "meh" here is unnecessary

[@firecatflameking]

actually a pretty good video

[@Vintagestep]

I'm don't really know much about AI, I've been following some projects, and I'm quite interested in studying and trying to come up with something at some point. For now I'm learning the ropes of web development to get a job relatively soon. With that being said, you raise a fair point by saying there's nothing really new happening on the paper, but comparing the outputs and capabilties of both iterations of this project we, once again, can verify that there's a big room for improvements just by tweaking fine details more than just the overview or the big things. So if anything this is telling me that it's worth the effort to maybe revisit some older models and incorporate recent developments, that if what we want is to focus on the output and the marktable aspect of this industry. And more as a consumer and part of the industry this is likely to impact(I'm a musician, but also I have close ties with illustrators), this tool may scare some people, and as impressive as it is, there's things that at least in this current iteration won't achieve, it seems to be aware more of the general aspects of the images than their details, composition, color, style, texture, lighting, it excels easily on all of this, but some aspects often show erratic results, when it comes to anatomy, faces are often a little weird, limbs, how things fold, like clothes or walls, there are good results but it's not uncommon to see erratic ouputs on this aspect either, so I don't think it'll replace illustrators or designers, it's more likely to affect the use of stock images, depending on the price as the narrator mentioned earlier. And I also think that while is able to replicate certain styles with jawdroppping results, there's a lot of rooms for the human touch and creativity to be explored, some illustrators I follow manage to stand out because they subvert some of the assumptions and rules this AI is taking for granted in a way that makes them stand out in a very artistic way, maybe this tool will make professionals and future professionals think more about these concepts and try to strive for a distinct style.

[@axcolleen]

The realistic monkeys ans astronaut is really photorealistic and if you told me that it's a staged photo I will buy it. This can replace some photography and artists who produce similarly to this is at a huge disadvantage.

[@Vintagestep]

@@axcolleen It might be perceived like that, but I don't think it'll put artist like that at disadvantage, the people able to hire them I think still would pay them, and this tool, while offering outstanding results don't have the same control to what you can get from commissioning a professional with the adequate budget, It will be used by people with less access to hire an artist probably. Maybe some people will change if they don't care but I don't think it'll be the case, think about something that's going to be printed for an ad in some place like subway or something, you don't want to have erratic outputs, if in some iteration it gets to the point where imperfections are rare and we can obtain exactly what we want, maybe, but still not everyone has the taste to produce something effective for what they need.

[@BusinessWolf1]

fuck the haters, shit is great

[@CharlieBrown-tr4zn]

needed a sober explanation since i came across some outrageous tech hype video on another channel that basically said all artist are now out of work... and then you end on a similar note. huh. thoughts on source code or replicability?

[@zander3456]

I thought so as well , but it's not really the case , all images we see are very cherry picked and created in a controlled environment, also the copyright situation is not clear , nor it will be able to be used in commercial situation. Ah btw Elon Musk is at the board of director and the projects he owns usually are marketed extremely well, but usually under deliver

[@CharlieBrown-tr4zn]

@@zander3456 yeah, that puts it a bit in perspective. still, before reading into it i imagined it was just recombining parts of learned images but the diffusion model definitely does a lot more than that - and apparently all of that is not even new. i think mostly i'd like to know just how much of a controlled environment all of this is hence my questions. dunno about copyright but i seem to recall ai research uses scraped datasets all the time without attribution (blew up a wee bit with facial recognition some time ago) so yeah, sadly that does not seem to be much of a concern.

[@Colorado-Coyote]

12:10 my favorite part

[@stacksmasherninja7266]

I absolutely agree with your conclusion though. Nothing novel presented. We know that results are VERY impressive but it's very hard to confirm that some of the generated images aren't from train set. Furthermore, none of the papers (DALL-E/DALL-E-2) studied whether generated images were truly original. They could've performed a CLIP search with a given prompt in their dataset to see if similar images exist in the dataset itself. Furthermore, uniqueness of the prompt ? We have 0 clue on how the dataset was collected. Plus, given its scale, how are we to believe that every text prompt they put in the paper was sufficiently novel ? How can we make sure that model isn't simply overfitting on it ?

[@EdanMeyer]

I forgot to check if they check for images being present in the training set, though I will say at least that lots of related research seems to suggest that these methods are not just overfitting. Of course when you start to deal with such large datasets confirming this becomes harder because simply checking for data leakage becomes harder, but with the caveat in mind the methods seem to be working well. It would be interesting to see though how well these models function on generating images of objects as a function of how many related samples were in the training set to see how far they are able to extend to the training distribution.

[@KeinNiemand]

I'm lucky enough to have gotten acces somehow I got acess after only a week of waiting.

[@shrimperlincs3395]

Has anyone asked Dall-e to Draw itself?

[@shinunotenshi]

Work on your titles, but otherwise great job!

[@Nerdimo]

Phenomenal 👏🏾

[@HorrorTactico]

Just came here to dislike for the "Meh". Not everything is about you.

[@robertchansky439]

has anyone thought of the potential for this as a memory aid? There's this idea of the memory palace, an imaginary place you go to in your mind to remember things. you've previously memorized associations between letters/numbers and images, and to remember something, you place a story-like confabulation of those images into a room in your memory palace. It takes a lot of training to do this, but people report abilities to memorize long strings of digits. this thing could help with coming up with the images. Just a thought.

[@edeneden97]

Thank you very much! very well explained

[@EdanMeyer]

Thank you, similarly named person

[@ogcurated]

Your channel has 6~ k followers relax with the meh lmao. It drew over 1000 images from WORDS in minutes in 1024x1024 pixels. Goodluck doing a 10th of that.

[@smartjackasswisdom1467]

Your videos really put into perspective how advanced are these models without all the hype. If you make a video talking about brain computer interfaces and their potential to use the neural data to create new ways in which artificial intelligence could be developed in the future that would be awesome 😎

[@EdanMeyer]

Happy you liked the video! I don’t have anything like that planned right now but I find BCI work very interesting and have touched on it in the past on my channel, so the possibility is certainly there!

[@pauljs75]

Imagine if that ever gets paired with a user-friendly UI and then gets in the hands of shippers and fan-fic writers... As if Twitter, DA, or worse parts of the internet isn't crazy enough, and the potential for drama could be hilarious.

[@ryzikx]

well so far dalle 2 tries to filter explicit content but apparently there are some loopholes.. we'll see 🤔

[@elZ-n2s]

@@ryzikx the horse sleeping in a pool of red liquid incident

[@rashid8646]

Everyone who makes clickbait contrarian titles like this one have the same low t voice

[@TSK0209]

Meh*

[@NuclearVEVO]

Deinp Learpt

[@TheRainHarvester]

Draw "frame 1 of a movie about ants". Draw "frame 2 .....

[@weighttan3675]

Frame 2: entirely different species of ants Frame 3: another species of ants different from previous two And so on

[@TheRainHarvester]

@@weighttan3675 "draw frame 2 in the style of frame 1" who knows. But yeah i had that thought too. Using the image tokens, i dont know how easy keeping the same motifv would be, but i think they could figure it out.

[@ChaosApostle]

Pretty clickbait. It's "meh but looks real"Maybe your opinion will change after you actually use it.

[@zander3456]

I'm a pro artist and was scared shitless for a day or so lol. However I did my research as well and ... The whole thing seems just to be marketed extremely well , but it's much more limited that someone would think . Also this is just a copyright lawsuit action in the making

[@youareasock9752]

Accshully it's meh🤓

[@dmarsub]

@comment section, ofc he means meh compared to the hype.

[@MochaRitz]

Hey, he put the stupid title, we're gonna point it out

[@henrykwieniawski7233]

So much of this video sounds like a word salad to me because of my very rudimentary understanding of AI/machine learning, lol. Super interesting, though!

[@sayamqazi]

A day will come when we end up with a human brain level of AI model and realize that our imagination is not that special at all. That would be a very very sad day to be alive.

[@Acrosurge]

Conversely, we might discover that there is an intangible spark of emotion or unquantifiable "soul" breathed into human art. AI can create the abstract and technically beautiful work, but can an AI image transfer the emotions of an artist to a viewer?

[@sayamqazi]

@@Acrosurge although i would love for that to be the case but the fomer option seems more likely.

[@EdanMeyer]

Or you could look at it the other way around, our form of imagination is so special and powerful that it can recreate itself via non natural methods haha

[@andrewkelley7062]

at 13 the duality problem,

[@plexoduss]

Great piece, a sub well earned!

[@muhammadamine6059]

Thanks

[@alexyz9430]

You're welcome, Kyrie. Good luck with your business!