Claude vs GPT vs o1: Which AI is best at programming? | Cursor Team and Lex Fridman

[@LexClips]

Lex Fridman Podcast full episode: kzbin.info/www/bejne/pXfJh6dpiJpkipY Thank you for listening ❤ Check out our sponsors: lexfridman.com/sponsors/cv8059-sa See below for guest bio, links, and to give feedback, submit questions, contact Lex, etc. *GUEST BIO:* Aman Sanger, Arvid Lunnemark, Michael Truell, and Sualeh Asif are creators of Cursor, a popular code editor that specializes in AI-assisted programming. *CONTACT LEX:* *Feedback* - give feedback to Lex: lexfridman.com/survey *AMA* - submit questions, videos or call-in: lexfridman.com/ama *Hiring* - join our team: lexfridman.com/hiring *Other* - other ways to get in touch: lexfridman.com/contact *EPISODE LINKS:* Cursor Website: cursor.com Cursor on X: x.com/cursor_ai Anysphere Website: anysphere.inc/ Aman's X: x.com/amanrsanger Aman's Website: amansanger.com/ Arvid's X: x.com/ArVID220u Arvid's Website: arvid.xyz/ Michael's Website: mntruell.com/ Michael's LinkedIn: bit.ly/3zIDkPN Sualeh's X: x.com/sualehasif996 Sualeh's Website: sualehasif.me/ *SPONSORS:* To support this podcast, check out our sponsors & get discounts: *Encord:* AI tooling for annotation & data management. Go to lexfridman.com/s/encord-cv8059-sa *MasterClass:* Online classes from world-class experts. Go to lexfridman.com/s/masterclass-cv8059-sa *Shopify:* Sell stuff online. Go to lexfridman.com/s/shopify-cv8059-sa *NetSuite:* Business management software. Go to lexfridman.com/s/netsuite-cv8059-sa *AG1:* All-in-one daily nutrition drinks. Go to lexfridman.com/s/ag1-cv8059-sa *PODCAST LINKS:* - Podcast Website: lexfridman.com/podcast - Apple Podcasts: apple.co/2lwqZIr - Spotify: spoti.fi/2nEwCF8 - RSS: lexfridman.com/feed/podcast/ - Podcast Playlist: kzbin.info/aero/PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4 - Clips Channel: kzbin.info *SOCIAL LINKS:* - X: x.com/lexfridman - Instagram: instagram.com/lexfridman - TikTok: tiktok.com/@lexfridman - LinkedIn: linkedin.com/in/lexfridman - Facebook: facebook.com/lexfridman - Patreon: patreon.com/lexfridman - Telegram: t.me/lexfridman - Reddit: reddit.com/r/lexfridman

[@BoredAsf-ji5rc]

I listen and pretend to understand

[@soheilarousta]

😅

[@gepvpr]

Fake it till you make it

[@frozenwalkway]

Ask chat gpt to explain the transcript

[@SalsaNinja13]

@@frozenwalkway god this was too funny

[@SouwaBeezy]

Hahahahahahahaha I’m listening to get some nuggets but man I’m lost!

[@asimdeyaf]

Unlike ChatGPT, I rarely have to read the code Claude makes. It often simply works the first time… plus it’s way faster.

[@vncstudio]

Yes. It generally works the first time with Python, Go, or C. With Nim, it takes a few tries.

[@petemoss3160]

@@vncstudio lol, i actually explored/learned some Nim via Claude Sonnet ... its a beautiful language in concept, but from what i understand there are still weird errors in edge cases, so for C compatible low-level programming i'm just going with C++ (with smart pointers for memory safety) ... but wow Nim is a neat idea!

[@الحمدللهعلىكلحال-م7ص]

every model has own way to talk with

[@meko264]

01 preview just built me fully functioning web analytics software. GDPR compliant, anonymized user ids, encrypted data, and offline viewing with python. I have limited space on my hosting account so I needed a web analytics software solution so I could monitor the performance of my pages without contributing too much to my file limits.

[@theaiffice]

How much experience do you have? How long did that take to implement?

[@meko264]

@@theaiffice - It was basically one shot, but I found that I wanted a few more features after I realized that it was actually working. I have zero computer programming skills. I made no edits to the program. The program is up on Github, it's called Aida Visitor Logger. I had to uninstall Matomo because my hosting has decreased the file limit drastically. Both the number of files and the total space in GBs.

[@meko264]

@@theaiffice - KZbin deleted my response.

[@MrWizardGG]

Using a CLI tool like Aider Chat o1 is better, and very easy to use.

[@meko264]

@@theaiffice - I have no programming experience. If there is any problem with the code at all I'll have to tell it try again and maybe give the error message.

[@tozrimondher4250]

I tried both. It’s Claude

[@nexicturbo]

O1 is better. Scored like in the 80th percentile in hard code force questions, no other LLM is that good except for o1. O1 isn’t even fully released yet, maybe you can debate with preview and sonnet but not fully released o1

[@Tokieejke]

With chatGpt I got feeling that it becomes more stupid day by day, with Claude it was like a fresh breeze

[@Dan_Campbell]

That's the one I use the most.

[@GermanCoder42]

Same.

[@GermanCoder42]

Well, you have to use it to know it. Sonnet is able to maintain context without losing it. Currently, I'm handling about 30k tokens per request. It's best to stick to the original plan. GPT sometimes hits the sweet spot but can occasionally hallucinate like a lunatic. Don't trust benchmarks; they're all messed up. Test in production instead. Here's a funny thing: Gemini is terrible at coding but remarkable in math. So when the problem is purely mathematical, talk to Gemini. Then use its solution to write code with Sonnet. As for GPT, it's an option if you have a lot of money to invest.

[@kdelmonten]

Bro the one to the right of Lex is doing his best Sam Altman voice impression 🤣

[@gregd6022]

That nasal just woke up choppy/raspy voice is not uncommon in the States.. they're all doing it in this vid.. they all live in the same bubble with Altman.

[@Chris-se3nc]

It’s this annoying generation of speakers. Born on the web and social media.

[@CollaredConsulting]

Perplexity let’s me alternate between them for the same question and Claude is best, especially if you train it using Perplexity’s Library feature and it gains expertise in a particular subject area

[@MrWizardGG]

We are talking for programming. Like using Aider or Cursor.

[@jokelot5221]

I use Claude and GPT 4o mini(because its fast), but i alternate between models.

[@SalsaNinja13]

@@jokelot5221 do you use o1?

[@jokelot5221]

@@SalsaNinja13 I tryed o1 mini, but for me it makes little difference with Cursor because i mostly use coding assistants as a way to speed up my development using patterns that i already coded myself. When used in this way, any model past GPT4 can be a great asset if you are looking for ways to boost your productivity and you are experienced programmer already. I use o1 preview on a chatGPT website when i need to do some brainstorming and as mean to bounce back ideas, and when im stuck with a particular design problem. For Cursor, Claude is really good for some more complex coding tasks, and mini versions are nice if you are looking for some quick edit or you cant remember some css attribute or a property.

[@inthegrail5998]

is there an English translation available?

[@tejasaditya551]

Claude by far

[@MrWizardGG]

O1 by far. Also shootout to Aider.

[@Manas-x1y]

As cs first year scares me Doing 4 years of engineering Worthless i don't know what to do in life Zero motivation Only anxiety , depression at its peak Life is becoming hell and hell more

[@rafaabreu5563]

1 of 4? Man you have more than time to figure it out, just focus on things most people dont like to do like emmbed, ai, computer vision, crypto, networking, cybersecurity, etc. The job market is just being hard mostly with web/app developers

[@thelisse555]

Hang in there friend! When you’re in a low, just know a high will come. It’s always a cycle and you just have to make it to the next high.

[@anthonybongo94]

It's another bias variance thing. If you can be super specific, o1 is the best hands down. If you are kind of unclear to what you want, then 4o is probably better. For me, o1-mini is just amazing. It will always write what you want, no doubt. You just have to be clear about what you want.

[@hbela1000]

awesome team, very much impressed by these young genious guys.

[@minghuiyu2073]

Claude's project feature is great.

[@Hemebean]

For me, o1 has been the most useful, by far for long and complicated scripts.

[@DrummerSi]

I have to say, I've been using cursor for a week now and I can't agree more with what's being said here. I think the point is, it's like talking to a competent programmer who gets your dodgy explanations (or implicit intention) rather than you having to prescribe and talk the lingo of an all knowing super AI programmer. I'm producing more work of better quality by a factor of 2/3x. Well done team

[@djpuplex]

Waiting for Agentic capabilities to come like Jensen said type in a prompt and get a fully functional program.

[@WimpyWarlord]

Been using both for last 6 months. Claude is better for coding.

[@sigmaroll9802]

Both are amazing, use both

[@davefellows]

I really want to see a comparison with o1 pro

[@KhalidAskar]

I’m nodding like I actually understand

[@u10-bb5lf]

from my own perspective, i think chatgpt is still dominant in the coding areas as well as in language model. Claude might do the same resullts or can be superior in programming but it has a problems understanding your requests. I honestly felt shit using claude

[@numb980]

I use both and can't pick a winner. But you can't go wrong with either one

[@0xBerto]

Also, o1 on cursor seems nerfed or duct taped. When I use I outside of cursor is doing laps on other models with code

[@MrWizardGG]

Use it with Aider.

[@Dionysus-Reality]

Well, yeah its super expensive the 50 per week we get on ChatGPT is with the thinking tokens nearly maxed out, whereas other companies are cutting down the thinking tokens in order to save costs and the like.

[@0xBerto]

1:01 eh I’d like to argue against that hahah. Build a complex react app and it’ll start shitting itself. But not horrible. Definitely can’t wait for the models a year from now to realllly rip out apps fast

[@almor2445]

What is it with tech guys and vocal fry?!

[@sgttomas]

i had to stop listening because it was so irritating

[@almor2445]

@sgttomas same here. So hard to concentrate when people do that!

[@mxolisimatha7175]

😂😂😂 I thought I was the only one that noticed!!

[@JohnGodwin777]

They all want to sound like Sam Altman

[@Alex1611AD]

The combined IQ in that room is probably over 700

[@gregd6022]

Hahaha, you're to easily impressed.. they can barely articulate themselves.. they are average "coding bros".. the type that AI will toast. Yeah i'm in the industry...

[@Alex1611AD]

@@gregd6022 whatever you say lol

[@dislikemike1151]

*800

[@lmnts556]

@@gregd6022 I would not call the 4 people to go from cursor's team on lex's podcast your average coding bros.

[@Rupert4090]

Dunning Kruger….

[@jaivan30]

What matters is (Agent Smith voice) is time to ship goes way down and more people can do it. Napster says hello pirate.

[@Darragh1]

I used to hate coding when I was learning it. Like I hated it, and I just remember saying someday....tehre'll be a program that does all this for you and then people will see how stupid it is that we have to 'learn' coding in order to do it. (akin to a calculator in maths) and now here we are.

[@gligom]

Sometimes AI is spot on, knowing what I want, I am amazed how right writes code. I just strat typing and sometimes he “knows” what I want and fill the screen with all the methods that I need. I just look at that code and cannot believe how good it was. This is just … sometimes. In majority of the time he makes so many mistakes and give me 100 lines of garbage code for a simple task, that i write in 5 lines of code. He just invent methods and classes that don’t exists in the framework that I use, and when you tell that is wrong … politely apologize 😂. I know from now on will become better and faster, is still young, only 2 years old, imagine at 5, or 10… Lucky for me then I will be retired 😊

[@therainman7777]

Sounds like you’re talking about autocomplete via Copilot or Cursor Tab. Autocomplete is cool but is nowhere near as intelligent as using a proper frontier model like Sonnet 3.5 or o1. I’d suggest trying one of those if you haven’t already.

[@gligom]

@@therainman7777 I use them all, for some time. All have the same behavior, some time magic some time junior dev 🤣 Anyway they will improve very fast…

[@JoeCryptola-b1m]

Rust I think he means which is the best at rust smart contract development that's what lex means

[@Splash滴]

With all these AI tools and use cases in “assistance”, I wonder how our ability to think will change overtime.

[@CV-wo9hj]

Claude handsdown

[@Youdude2]

Only way to find out is create terminator robots and let them fight using these LLMs

[@IvanRandomDude]

Claude.

[@HalkerVeil]

I thought Lex was an actor that started a podcast. Am I tripping?

[@Amulya7]

You are

[@alst4817]

He’s an AI

[@omarhabib7411]

he's a computer science PhD from MIT

[@Julian-pg9dx]

@@alst4817 You're confusing that with the Zucc

[@Kurushimi1729]

​@@omarhabib7411he worked at MIT. Didn't get his degree there

[@dannyGadventures]

Sonnet❤

[@thiagoracing]

Trying my best to pay attention to the content but hard as they mostly rumble and say nothing apart of Sam Altman impersonations 😅 Lex seems to be the only one with his own personality

[@surfingbilly9654]

Why is the Cursor Team being treated like AI experts? They literally forked vs code, paid for an openAPI key, copied an existing products code (github copilot) and called it their own, with temporarily cheaper prices funded by VC. It's basically theft at this point.

[@JohnGodwin777]

What’s your product that’s better?

[@surfingbilly9654]

@@JohnGodwin777 github copilot

[@slawomirgoleniak]

Thumbnail say 4o and title say o1

[@STM-n4t]

Lex always looks and moves like he is high has a kite.

[@Dancoliio]

everybody sleepin on Grok 3... elon boutta enter the chat real soon

[@freedom_aint_free]

o1-preview blows Sonet 3.5 out of the water

[@1211]

Nerds need to learn how to explain to the masses.. Well let me try Claude one more time but GPT4 has evolved well btw. It gets the code the first time.

[@JoeCryptola-b1m]

I feel like money is standing in the way of logic in computer science world I think it got messed up where binary met higher level languages this is where it gets stupid & stops following rational flow of processes

[@punk3900]

o1 is the best but it's context is lousy

[@IrakliKavtaradzepsyche]

Claude

[@Blueprint4Murder]

Those programs don't actually program it is code piracy. So the correct question is which is better at identifying which scenarios to identify the right code to pirate. That is the problem with AI because if you have to go through 20 searches to get a working code its probably faster just to know what you are doing. The next problem is when it only puts out that code for that scenario which means every program will be the same, and the next problem is security because if every programmer uses the same code it means every programmer potentially has the same security issues. So the Idea that AI is going to take over code is silly, but with the computational power and machine learning maybe it can create more efficient coding languages on one or both ends.

[@rmdashrfv]

Yeah that guy said a lot of fancy stuff to basically mean this. If you present it with a scenario it hasn't seen before it's absolutely worthless, all of these models

[@SalsaNinja13]

I’m not exactly sure your background in coding but it sounds like you have an understanding. Coding is much more nuanced than the way you describe. Coding is a series of tasks and instructions on how to solve them. Similar tasks usual have similar solutions. The majority, like vast majority, of programmers do not come up with 1000s of lines of code from scratch. They use similar logic, algorithms, and even repositories of code snippets. Many programs will have recognizable and similar parts, but it’s in the implementation of all these parts that make each project different. AI struggles with implementation. It can’t really conceptualize what someone wants, and can only crank out a few dozen usable lines at a time typically. Anyone who’s tried to code with chatgpt specifically knows that it can take 20-30 prompts to get a usable feature. Where AI shines though is it’s ability to quickly code smaller tasks. It also does a good job at suggesting fixes or “unclogging pipelines”, but even that requires the users to enter in detailed and descriptive prompts to work properly. Where it is right now I think of it more like an assistant to a chef. I could do everything myself while cooking or I could have someone help cracks eggs, chop onions, and wash the dishes. Outcomes the same, just one ways more efficient. As it gets better and eventually get to a point where it can write a 1000 lines of code in 10 seconds, then piracy and open source issues will become a legitimate concern. But the issues will revolve around the ethics of it, and how willing these companies are to share their models reasonings and show how each response is generated.

[@Blueprint4Murder]

@@rmdashrfv Its also worthless with to much data, because then it will give the same solution every time. Which presents more problems than it solves. Not only are there issues with a correct good intentioned training they could be trained maliciously.

[@therainman7777]

No, they actually program. If you think it’s just “piracy” then you haven’t spent nearly enough time working with them. Also you can’t call something “piracy” if it was performed using data that was willingly made public. Only copyrighted material can be pirated. If it’s not copyrighted and it’s released to the public, then it’s public domain and piracy is no longer relevant.

[@therainman7777]

@@rmdashrfvThat’s not even remotely true. I present these models with strange use cases on a daily basis that would never exist outside of my business-you might say I couldn’t know that, but I guarantee you because of the nature of my business, I know it. And 90% of the time the models give me exactly what I’m looking for. And I’ve been programming for 30+ years so it’s not because I don’t know how to evaluate code.

[@julesripoll2397]

what's up with the vocal fry

[@TheSSEssesse]

They all work in Southern California.

[@JohnGodwin777]

Ikr.. 🐸

[@verigumetin4291]

I am disappointed at how stupid some of the commentors to this video are. Truly disheartening.

[@elielmathe]

So do you really prompt jsx 😂😂😂 Wtf Lex?

[@rubyciide5542]

Claude just dominates the game are u blind mr desi All talk

[@ChristopherPhillips]

Why do most AI people have an incredibly annoying way of talking - not just the accent, the unecessary selection of vocab.

[@gregd6022]

it's not just "AI ppl", it's "coding bros", it's a "self-fulfilling prophecy", they can't function in most other environs (very low Social IQ), they can barely communicate here in their own field.

[@vidil-h5i]

revnge of the nerds

[@3LITA]

Use AI to get rid of the super annoying and pretentious Vocal Fry!!!

[@vatsrajat23]

Why does everyone in US have a soar throat

[@edwardsanchez3708]

If AI is so smart and has all the information ever put on the internet at it's dirty little cyber finger tips, how come it can't come up with new cures, ways to balance the budget or new technologies?

[@stimu_li]

Probably has something to do with the fact that these models are mostly language-based. Can new cures be developed purely by reasoning with language and written information? Maybe. Is it harder than performing experimentation and using the scientific method? Probably. Good AI is able to synthesise information from many multiple domains accurately. But, as far as I understand, when you ask a language model a question, it's not referencing the entire archive of its training data and performing synthesis to create new information; it's still sorta just trying to do a good job at predicting its own next word, lmao.

[@justlearning1611]

Oh it definitely does/has. Us little folk just don't get told about it

[@MrHaggyy]

Well it can work combinatorically. Which to be fair is a very solid drive in research and innovation. But as now it can't be truly novel. It's ability to reason about the scope and compute of a project is also very limited. Also the "AI" we use today to synthesize proteins to get a cure, or the optimization algorithms we use for wing geometry, or the magnetic flux or combustion in an engine are vastly different to LLM's. In budgeting it really helps in dealing with technical depth, as reiterrating with the execution speed of AI is even cheaper. Which isn't necessary a good thing, as driving for even more mediocre products and updating them last minute to the bare minimum requirements gets even more profitable compared to providing products with real value.

[@CollaredConsulting]

Why would you think they would tell you if they did?

[@RHall1]

While it is true that if “cures” were discovered, they would be kept secret, the vast majority of the “data” on the internet is cat videos and idiots twerking. While a lot of information can be gleaned from that data, very little of it is useful in curing cancer.

[@raijin__ow4426]

chatgpt is too lazy for programming in 99% of the cases

[@SiLoMixMaster]

If any of those dudes voted for Trump I'd be shocked

[@Charlie-Em]

Horrible physiognomy all around that room.

[@lem00na]

mistral large

[@dazeb]

Neither directly, Bolt.new is the new champ 👑

[@subhamnayak866]

bolt uses claude .

[@ashleigh3021]

No chance

[@gaivoron]

They all got open pen and paper lol nerds

[@JohnGodwin777]

So much vocal fry in AI

[@Teting7484f]

Ai scammer

[@Aditya-m5k]

Why do you post videos with such banal questions, Lex? Go read Heidegger's critique of technology. You could ask even better questions.

[@gregd6022]

You don't understand his objective... nor should these goofy yoyos who've never been outside their dorm room be asked anything beyond their "banal" lives..

[@b8pvz]

Claude