Huge ChatGPT Upgrade - Introducing “Canvas”

Here's a breakdown and some testing of ChatGPT's new Canvas feature.
Try Vultr by visiting getvultr.com/berman and use promo code "BERMAN300" for $300 off your first 30 days.
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewber...
My Links 🔗
👉🏻 Main Channel: / @matthew_berman
👉🏻 Clips Channel: / @matthewbermanclips
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
👉🏻 Instagram: / matthewberman_ai
👉🏻 Threads: www.threads.ne...
👉🏻 LinkedIn: / forward-future-ai
Media/Sponsorship Inquiries ✅
bit.ly/44TC45V

[@matthew_berman]

Will you use this feature? There are a few more days left to win the Dell Monitor in our giveaway: gleam.io/q8wkK/dell-nvidia-monitor-2 NA only

[@Josephkerr101]

Already have. Be warned about broad rules based changes on large text blocks. I had a handful of mis capitalized sentences and it went crazy after asking it to "go through the text and correct capitalization errors".

[@wessel175]

I am actually super excited. This will make it soo much more easy to use. I remember my frustrations about trying to write out my thoughts and how chatgpt kept missing details. I feel like this feature will make it easier for me to add and change. And the models themselves should also get better at paying attention to details and formulate with less mistakes and asking the user more and stuff like that

[@philatelist]

Unfortunately it actually can't handle large code snippets, after a few code changes (about 500 lines) it starts to truncate the output inside the Canvas and there is no button like "Continue Creating", chat sure everything is ok. So I had to work in the old mode.

[@jeffg4686]

You knew it was coming... AND They STILL cannot see socialism yet

[@technocorpus1]

I found what you said about platform companies to be very interesting. Even though I super like AI, maybe that was my favorite part of this video!

[@youtou252]

thank you for not making a cut every 2s, this makes watching your videos much more enjoyable !

[@RupertBruce]

I couldn't help but notice some blinding transitions though...

Agree

[@RadiantNij]

That's why I like his videos something about being involved in interacting with the content instead of doing a voice over seems better to me.

[@RadiantNij]

​@@RupertBruce you need to reduce your notice sensitivity from 9 to 2 lol

[@bring_everything]

8:13 the the mysterious hand appears over you shoulder, love it

[@c1ark3c]

Great, my 4 yr wanted to see this over and over.

[@signupisannoying]

This video is AI generated

[@OriginalRaveParty]

Platform risk is everywhere ... If you build a large Twitter following but then get banned. If you run an eBay business and your account is closed. If you work on any of the freelancer platforms. If you are a popular KZbinr and you get shadow banned or shut down.

[@8483na]

this is just fire.. thanks for keeping us updated in the rapid exponential growth of AI space.

[@I-Dophler]

Key Takeaways: 1. OpenAI has introduced "Canvas," a new feature designed to improve collaboration with ChatGPT, particularly for writing and coding projects. 2. Canvas offers real-time editing by allowing users to open a separate window where they can request targeted changes to text or code, moving beyond a single chat interface. 3. Currently available to pro and enterprise users, Canvas will be extended to free users once the beta phase is over. 4. Canvas is tailored for both writing and coding, offering features like editing suggestions, adjusting reading levels, fixing bugs, and converting code into different languages. 5. GPT-4 has been specially trained to recognize when and how to use Canvas for these tasks, enhancing its effectiveness. 6. The video demonstrates Canvas being used for testing, bug fixing, adding logs, and creative writing, including making edits like adjusting length or complexity. 7. The creator highlights a potential concern for companies relying on OpenAI’s platform, as Canvas might pose a competitive challenge to AI-based businesses built on its technology. 8. During testing, Canvas is shown to handle bug fixing and log adding but occasionally modifies more code than necessary. 9. It also supports creative writing by offering options for refining text, adjusting tone, and even adding emojis or final polish. 10. Viewers are encouraged to share their thoughts on Canvas and the potential platform risks for AI companies dependent on OpenAI’s API.

[@cycologist8615]

Nice AI summary!

[@I-Dophler]

@@cycologist8615 you are very welcome

[@TheFelixAlabi]

AI did this

[@I-Dophler]

@@TheFelixAlabi A human got back, mate.

[@bigpickle8844]

Canvas is a huge step in the right direction! Can’t wait to see what it is in 12 months!

[@punk3900]

Great comment about the risk of using platforms! Thanks for sharing!

[@sascha-born]

The platform bit in the beginning truly should be on everyones mind - providing "just an API" and see the internet build on top has the highest risk of over-night competition anywhere... You build in secret everything that you know your API is asked and as you enhance your API, you know what works and what doesn't.

[@noxnox2172]

True words regarding "platforms"

[@simonagamero4947]

Another great update. Thanks for this! 🎉

[@MelindaGreen]

The trick to reducing risk of platform changes is to always create your own API that sits between your code and theirs. Then you write some simple test code using your API that only lightly tests the basic features, and make sure it works. This is not a replacement for unit tests. Rather think of it as part of your API. Finally, implement your own version of that API that produces the same results as the platform you are wrapping. If the platform ever disappears, you will have a fixed target for the functionality you need to recover, and can easily farm out that work if needed. And perhaps best of all, even if you never need to move away from the platform, you will have an easy way to prove who is responsible when your product suddenly breaks. If it's still broken when you use your test implementation, it's your bug; but if that fixes it, it's the platform's bug and you can prove it without even giving them your code.

[@PH-xl6nz]

The new Canvas UI and some of the functionalities are very similar to an investigation AI I saw the demo for called Kolabrya. They use chatgpt api. I think platforms tend to stifle innovation when they compete with businesses that flourish in their ecosystem…

[@TechNinja_420]

the thing you mentioned, that OpenAi would steal the data and be a competitor, is the exact same thing, which amazon has done in the past and is still doing, copying and rebranding ..... so yeah, the fear is justified

[@NickDrinksWater]

I love ai so much, especially knowing how fast its improving!

[@punk3900]

Canvas is a great update for a starter. I'm looking forward to re-working the whole context window.

[@MrElectrecity]

As for the platforms, you are right, startups cannot survive by relying on giant companies, and this is what happened with OpenAI, as since the launch of its products, it has been monitoring small companies that use its products to build their platforms and began integrating these ideas into ChatGPT or working on an API from it at a cheaper price.

[@DanFarfan]

I'm skeptical about canvas (though version 1 has a few good parlor tricks). How many apps are realistically 1 file? When I wrote my screenplay(years ago), I worked in a file per act. Until the very end. Curious if canvas can handle the hollywood screenplay standard format (which native gpt can't). More anxious to get my Pythagora account turned on. But if that doesn't happen soon... Keep up the good work, MB!

[@kamsolt00]

In terms of risks. I think it depends on what you build. If you just add AI capabilities into your own application that is just enhanced by it, you're fine. But if you want to build an app when you literally just chat with it, I don't see much future

[@AIPulse118]

Claude 3.5 sonnet is still the superior coder. This is a band-aid to drum up excitement. The underlying model is still 4o and still less adept at coding than 3.5 sonnet.

[@jerikho04]

From personal experience with topics related to Cloud, Orchestration and Containerization, ChatGPT 4o is far superior to any version of Claude, imo.

[@erkinalp]

let's see how claude 3.5 opus or 4 opus will behave

[@iwasthererecording]

I’m paying for both Claude and ChatGPT. They both have their strengths. Claude certainly has impressed me. If I was going to pick one I would choose ChatGPT. I use it mainly for writing code.

[@OumarDicko-c5i]

Other plateform like cursor should finetune their own model or else they will lose

[@thesimplicitylifestyle]

Well... Time to write a science fiction book with Canvas! 😎🤖

[@leopoldomolina1763]

Hey! What are you up to? I am in the same mood! Actually some weeks ago I started to sketch something similar to this canvas feature for writing and now I feel a bit frustrated with this but it is what it is. Now hands on!

[@Earzone63]

@@leopoldomolina1763 I wrote first chapter for a writing competition and that was quite a good starting point to use.. just put it up. kzbin.info/www/bejne/iHXcinR-j6qoY6M

[@KennethEd]

Except the value of a science fiction book is now fractionalized. And if I liked your output even a bit I'd just plug it in and change it more to my liking on the fly. But go ahead.

[@thesimplicitylifestyle]

@@KennethEd 🖖🏽😎🤖

[@DailyTuna]

This building on platform in a way is why Zuckerberg is so intent of building his own platform after being out maneuvered

[@ai_outline]

This is awesome for software development!!!

[@Thewerwolf]

I agree Claude is still the king when it comes to code.

[@abd-elrahmanmahmoud3167]

it does generate the whole code again ( with new fancy fx that shows the part you marked changing in live mode ), just like Cursor, no new technology in here, it's same as you copied your whole code to gpt and asked it the same question, it will generate the whole code with ONLY the part you mentioned

[@josephflemming7370]

It’s the chance when you build off a platform companies technology stack. Sam Altman has been addressing this very subtly for several years now. It’s not just about features; it’s about the whole package that a company offers. Including presentation and customer service. Small companies needs to find ways to deferential themselves from the pack.

[@christophnikolaus3428]

I can see this getting expensive for them, for every single new change request it appears to me that their model re-reads the entire chat history and I can already see loads of people doing request over request without starting a new chat. One could already see the latency going up after several changes... They will either have to heavily restrict the maximum number of questions per day, heavily subsidise this or start a scheme were they charge you for each question. But maybe I'm wrong and they developed something that is able to kompress the old messages efficiently or they aggressively delete older changes (for the LLM not the user) and only bring them back if a user specifically interacts with them again. If I were a cynical person I would say they are working towards a future where users get hooked on those models and can no longer work without them and then the payday begins, I mean there has to be some reason why the entire ethical department left OpenAI together with many other key people in OpenAI. From the outside it looks like every person with a conscience leaves them and goes to competitors like Anthropic.

[@dhamovjan4760]

During a chat, at each message any llm repeats all the current chat history. That was the case before, and mostly the case now. Remember that an llm has no memory. It fakes memory by taking all chat input as the input. At one point the chat history gets longer than the max context and then its loosing stuff. One way to manually go around this was editing the last prompt and including the previous output instead of asking follow up messages. This new canvas is, in a way, also doing something similar. In a follow-up question, not all the previous history and canvas content needs to be input; it only needs the last version. So, in a way, this is actually cheaper. It takes in the history of the chat messages, but for the current code in the canvas, only the last version is needed.

[@tee_iam78]

Great contents, as always. Thank you. I may be that with Canvas built-in the system will use less token comparing to the classic chat conversation one.

[@lucindalinde4198]

Totally agree about platform risk. I've lived through the same storyline.

[@josephflemming7370]

It’s the chance when you build off a platform companies technology stack. Sam Altman has been addressing this very subtly for several years now. It’s not just about features; it’s about the whole package that a company offers. Including presentation and customer service. Small companies needs to find ways to deferential themselves from the pack.

[@dennis4248]

ChatGPT is becoming better and better. Very impressive

[@PZMaTTy]

That was quick

[@WildJag4free]

Can’t wait till they add a loop where it runs the code and debug it before it releases the code to you

[@rewdh]

How do you pretend it to run a code which is part of bigger solution?

[@WildJag4free]

@@rewdh ?

[@augmentos]

Good one enjoyed, Tnx Matt happy weekend

[@anta-zj3bw]

When is the desktop app for Windows going to be released?

[@jerikho04]

Thanks for the review, will Canvas be available in custom GPT's?

[@jonberrydotnet]

Your “change these colors” example made me realize I can take any hardcoded website theme CSS and convert it to CSS variables - allowing for dynamic pallet changes. Not an everyday need but just another use case for GenAI doing the tedious-to-the-point-of-impossible work.

[@garyhuntress6871]

I'm a big fan of formal software requirements. I'd really like to see that capability added. It should take a general description of what I want and make formal "the system shall..." or "the page shall...." etc. It should maintain that list and write the code against it. It should be able to write tests that verify the requirements, iterate when they fail, and refactor code if requirements change.

[@BlayneOliver]

Love Canvas 🎉 Coding is so much fun now

[@slainiae]

Hi Matthew. Can you do a video explaining Cursor please and why it seems to be such a big deal?

[@WildJag4free]

😮 that was what my face looked like during the entire video

[@stal1963]

I don’t want to be dependent on a specific vendor API. That is what I learned in the last 3-4 decades. However, when AI coders are strong and powerful enough, they could provide code for building a wrapper around these APIs or introduce a more “generic” API that can be mapped to different vendor APIs. Currently, I try to implement LLM applications in such a way that I can easily switch to other LLMs/APIs. Most process steps are exactly the same. I am curious how Pythagora solved this issue. You never know the strengths and weaknesses of LLMs in advance. Thus, independence is a great value.

[@brigfiche]

OMG canvas is soo good. However using gpt-4o with canvas has way less reasoning ability than o1-preview. I tried making a python program and canvas/4o was consistently not able to correct a mistake where it called a function before it declared it. I copied the program into a new instance of o1 preview and it fixed it immediately and like a boss. I liken it to trying to explain a complex issue to someone with a different first language. I can't wait until o1 and canvas are combined!

[@Thewerwolf]

My previous company had the same issue platform risk it is real and the only way to mitigate continue innovating and focus on industry specific solution where the platform hopefully will not have the pace to keep up with you.

[@alexanderhdmi7704]

Does anyone know if this would benefit those who are looking to build a website with no coding knowledge? I know Claude can manage projects efficiently. Could I have multiple interacting code files, like one for java, one for css, one for html?

[@cycologist8615]

What tools do you use to manage your entire code base. GPT-4o and Claude can’t keep track of changes across multiple modules over time

[@deniszvegelj7651]

For me o1preview is much better in help with coding, so not sure if this will be better with canvas and 4o model. What do you experience?

[@sveindanielsolvenus]

You must tell it to edit the code locally, or to not rewrite the code. Otherwise it will run through everything, like it did in your video, Matt. Ask it to change or fix a bug, then also say "only local edit" or something along those lines. And watch the magic. Much faster.

[@shaunralston]

Great video.

[@randy1984d]

This is an awesome feature!

[@frankjohannessen6383]

So you can actually modify parts of the context that was generated by the LLM? I wonder if OpenAI has thought carefully about this in regards to Jailbreaking.

[@3thinking]

Note to self, build a platform company.

[@VictorGallagherCarvings]

The functionality may be basic but I suspect that for many of us, it will be all we need.

[@jewlouds]

That's actually sick

[@Tetsujinfr]

So this canvas functionality is a supervised fuctionality, where basically we are now giving even more feedback to openai to help them improve on what chatgpt does not do well enough initially. Let s see where that goes

[@marvinpolo3180]

if you'd be the wright brothers discovering human flight for the first time, would you stop for a week or two to make sure the seats are comfortable? Chatgpt O1 and Canvas and Voice interfaces etc. etc. tells me this is how good it's going to get. You wouldn't waste your time on those things if you'd be close to a breakthrough that will change humanity forever. I think there's a reason those features are released instead of GPT5.

[@totoroben]

How close are we to graphic designer end users being able to edit source code to make GIMP UI and features to be a useful alternative to Photoshop

[@joseantoniogomes6407]

They should go for something like artifacts especially for coding... But these are fun improvements

[@young9534]

I like it, but I want the ability to have different tabs open for different code files. Similar to how an IDE or text editor works

[@rolestream]

Vulture? Hmmm sounds like a company picking the carrion from the AI battlefield.

[@deniszvegelj7651]

so far this seems a mess to me. I've put 350line php form page, and asked to add a simple button to switch values of two fields, and it just skipped large portions of my code, and after several prompts restored lenght of the code but still parts missing and page wasn't working any more. For now I'll stick with o1, though it's silly to always reupload several files that need updating on every prompt, but at least is quite reliable output, much better than 4o

[@storiesreadaloud5635]

Does getvultr see what we're doing with the compute we rent?

[@SirajFlorida]

Wow, openAI decided that the way Anthropic does things is better. I guess the best for of flattery "IS" mimicry. :-D

[@bamit1979]

Artifact does not execute code. Only the code that can be shown as webpage or mermaid chart but not python code etc.

[@AIbyRakshit]

Your content is so good. Love from India 🇮🇳

[@mavieautomatique6645]

MOLMO AI would be nice if you made a video about molmo and how to get the x and y coordinates in an image.

[@TheAIPivot]

I loved your story about strawberries with intelligence 😝🍓

[@TheRagingUnprofessional]

I literally find out about the new features because all my ChatGPT tabs have to re-auth lol

[@Gothraga]

The Truth is Matthew was AI all alone!

[@WillBeebe]

Being able to switch to Turtle like that, with a UI like that, is nothing I've ever seen before. Ugh, I might have to use ChatGPT again...

[@BruceWayne15325]

OpenAI has moved to a for-profit company, so yeah, building on top of them is exactly like your experience building on an API company. Bill Gates has already said that in the near future AI models will be worthless. There will be so many options, and the physical constraints (energy / $ to train) will be so large that they will all reach the same level. The value won't be in the models themselves, but in what you do with the models. If OpenAI wants to make money, they will definitely use their software to that end.

[@isfk]

Pretty soon you'll be having AI make these videos for you.

[@ChefBrianCooks]

Grammarly just went out of business.

[@MichaelForbes-d4p]

It might be a good idea to discuss corporate governance laws. It's annoying for the platform to compete against their own customers but is it within the scope of governmental power to prevent them from doing it?

[@FunDumb]

Platform risk is at an all time high. If you aren't prepared to create and leverage your own platform, you're definitely at risk of going out of style.

[@DemetriusZhomir]

The UX design of the canvas mode here doesn't look right... E.g., language translation - why is it not a dropdown list?

[@leopoldomolina1763]

This is part amazing and part frustrating because this feature of a canvas is very helpful and has an enormous potential, however, it is frustrating for those small software developing agencies that come up with great ideas and are quickly surpassed or burned by open AI resources. What they are doing is to compete with openAI, at the end, from my point of view, small developing agencies will only be training playgrounds for talented people that in the future will be bought by openAI mergers

[@denisblack9897]

Im amazed how important animations are… its the same random bullshit generator, but looks like an ide, so people feel like its doing real software development😅

[@michaeltse321]

exactly the problem with cloud companies like aws, azure and google cloud

[@DabnisUK]

Another great video Matt. AI does not have imagination which is a key ingredient of Innovation and this is my fear, that such tools will stifel innovation. While it can be seen as a 'creative' tool it is limited by the bounds of what it has been exposed too. Once we reach AGI, my opinions may change. Platform dependancy, I agree entierly with your comments and would extend them further to include 'frameworks' that many of us use, although to a lesser extent. **REM: Humans are 'lazy' by design! I'm old enough to remember when people believed something was true just because it was in the 'newspaper'. The same was true in the early days of TV. How long before I hear 'but it was produced by OpenAI canvas', does that imply that it must be correct?

[@Blackhearts60]

I think relying on AI for creativity is kind of antithetical to the point of it. Usually people come up with the ideas (because we like to do that) and then AI helps us exexute them.

[@DabnisUK]

@@Blackhearts60 A valid point, but it does not lessen my fears of how such advancements will impact innovation. I repeat ** Humans are lazy by design in that most of what we create has an aim of making some part of our lives easier. It started with the wheel! not suggesting that this is a bad thing, but as a 'general' observation, given a choice humans will tend to choose the path of least cognitive effort. This leads to us using our minds less and less, and while some may argue that this frees our minds to think about other considerations, this will not always be the case. As humanity evolves and we develop technology that removes the need for congnitive effort/thought, where could this lead us?

[@98f5]

Was just playing with this its great

[@thirdreplicator]

Can it edit 2 files at the same time?

[@shuntera]

Matthew, Matthew, Matthew- does it actually run the f****** snake game correctly?

[@pruff3]

Tried it, can't render JavaScript or whatever in browser like Claude, lots of mistakes. Cooler than original but nothing special.

[@stuj1279]

Changing code when it is not asked to is definitely NOT OK... How can you say it is??

[@TheFelixAlabi]

Why is chat GPT suggesting edit to the text it generated itself?

[@serg331]

Currently, this sucks. If you don’t say ‘open the document’ it will insist that that’s not a feature in ChatGPT. If you ask how to open it in the canvas, it’ll also insist that’s not a feature. It attempts to edit what you want and fails, to the point where it can no longer edit anything anymore. I ended up starting 3 new chats because it wouldn’t edit and failed to let me open it in canvas too.

[@Jimmy-xt7jr]

nice

[@davidallred991]

I hope they update the Mac app to use it also soon. I like working in the Mac app much better than in the webapp.

[@drakefire98]

they should add the ability to exicute code

[@Björn-w7v]

In days of well integrated ai ide this is not a relly big deal. I think the reason why this is so hyped is that youtubers need to publish new content.

[@warrenb2856]

OpenAI is a massive platform risk! There is a reason why Google did not build on Microsoft Windows in the 90s.

[@badca52]

I like this new feature. You'd obviously like it to write the code as you want the first time but that's just not realistic even with humans. It got tedious and cluttered having to copy the code or sections of code back into your conversation without canvas. Good to see the conversation separate from what you're working on.

[@profikid]

They call it sherlocking

[@JR-kw6sd]

@matthew_berman I haven't seen anyone cover this yet but 4o has taken on a more inquisitive approach to answering questions ever since O1 came out. Instead of simply providing a one-shot answer, it's first response is sometimes a clarifying question. I'm not sure if this is a personal setting based on my managed memories or if it's a new standard that seems to be taking a page from O1 thinking. Have you recognized this behavior at all?

[@vodkaman1970]

Tried it and it is not much use to me it can only hold about 900 lines and doesn't work across multiple files. That's enough to write a snake game but anything beyond that it is less useful than normal prompts and code snippets.

[@HunterMayer]

It's like an out of control jr. dev...

[@arun279]

im concerned about model fragmentation. 4o series has canvas and advanced voice. o1 series has reasoning fine tune. they need to converge all their capabilities under one model.

[@laukmedina]

Maybe you can make agents that would make it.

[@leanfuglsang1320]

Having tried both, Canvas feels like a really bad rip off of Cursor. Except cursor can handle multiple local files.

[@TheTeachMeister]

One thing you left out that it can do is it can actually use o1-preview so it will "think more" EDIT: Type / before starting your prompt and choose "Reason"

[@drcharleyLGO]

😮 Can you explain how? Couldn't see that option

[@TheTeachMeister]

@@drcharleyLGO Type / it will show you options such as picture/search/ and the one your looking for reason.

[@TheTeachMeister]

@@drcharleyLGO You have to type / before starting your prompt and choose the option "Reason" there are other options as well.

[@Blackhearts60]

Wait are you serious? If you type "/reason" it will use o1?

[@TheTeachMeister]

@@Blackhearts60 yes if you do it before your start your prompt it will use the o1-preview.