AI Agent Automatically Codes WITH TOOLS - SWE-Agent Tutorial ("Devin Clone")

[@matthew_berman]

How many coding agents can you handle??

[@marilynlucas5128]

Imagine SWE-Agent using Aider! What do you think? Can you do a tutorial?

[@lancemarchetti8673]

It's a pity there are no coding Agents online as a webui for us folks with slow laptops :/

[@PrinzMegahertz]

Yes

[@joe_limon]

I just need one agent to handle all the other potential agents 😂

[@int_pro]

Yes, exactly. Was going to post and ask you, at this point which of all of these is the best for coding? (of those we can access... DIY ideally but corporate if that's clearly superior) Just a few too many to keep up with. Thank you for all the great content!

[@ofirpress]

Matthew- thanks for covering our work!

[@DailyTuna]

This AI news is becoming so crazy that if you miss three days, it feels like your months behind

[@ErinWallace7]

Exactly

[@aoeu256]

Hmm is there any AI program that can help me troubleshoot my computer by looking at log files, also I want an AI that automatically handles UI configuration of windows via OCR, also an AI wizard to get me API keys for various APIs and do web navigation for me. Also an AI to help me find people who want to share and develop AI programs on various social media. I wish I had an AI that could translate english messages to chinese for me.

[@phen-themoogle7651]

@@aoeu256 GPT3.5 (free) is native level in Chinese, according to my Chinese aunt who is a university Chinese professor. So I know it's legit with Chinese ability. Gemini Ultra and Claude3 are even better at languages from my understanding, at least in the languages I'm fluent in (Japanese). For my purposes I had to purchase Claude3 (I don't use GPT4 anymore since Claude3 is way superior at Japanese, I can write novels with it if I prompt it correctly). And for all the other goals, sounds like you would be better off focusing on achieving one of those at a time (if there aren't available programs) and learning how to make one via python with the help of a LLM at or above GPT4 level, Claude3 is amazing at programming, but might need to use it or another powerful LLM in combination with KZbin tutorials/forums/documentation etc Other people would also be interested in that technology or combination of tools if you can achieve it, if it doesn't exist already... so you could either sell it or open-source it. If you make a discord or something or find a good way to collab with other people on creating new technology I would be down to join (I'm only an intermediate-level programmer, but I'm resourceful sometimes, and finally got into a good work-flow for game dev) For Asian languages the OCR can definitely be improved, when I try top LLMs and various sites they always struggle with Japanese fonts if I just take a picture and have them go from it. So I would be especially interested in that if you can make or find something that very accurately transcribes Chinese/Japanese characters from a book or novel. www.perplexity.ai/ , sometimes is a really good AI search engine, not sure if it can help you find the info you need. Good Luck (⋈◍＞◡＜◍)！☆

[@DailyTuna]

@@aoeu256 you need just to clean install. Time is the most precious commodity we have. Sure it’s interesting to find and fix the problem if you have the time. On that other stuff, you’re right

[@csells99]

you don't have to wait for the swe-agent to add support for local LLMs; just log the bug, point swe-agent at it and bob's your uncle.

[@avi7278]

Bob is my uncle how did you know?

[@avi7278]

He also happens to be my grandpa...

[@kaiowasdeath]

Is Bob Martin your uncle, too?

[@zyxwvutsrqponmlkh]

Actually my uncle is Robert not Bob

[@s0ulweaver]

Bob is a builder. Wo hamesha karke dikhata hai.

[@TnotezProdutions]

Think we have been waiting for this for a long time

[@Saiyajin47621]

I’ve been following AI news for years. It’s like a real time ongoing competition right now, where different companies just kept overtaking each other. Super exciting 🎉

[@armans4494]

Mathew, your channel is a gem. I love how you always take these theoretical concepts and break them down to meaningful items. Not to mention, your tutorials/how-to guides. Simply the best❤

[@robertheinrich2994]

I guess, the next step is: have the LLM locally, finetuned with the codebase of linux (practically everything), and allow it to access the operating system, config files, etc. result: you have a computer that can run and modify itself. devs can go further and let SWE fix linux and the applications. essentially, copilot from microsoft, but it can modify the source code.

[@EduardsRuzga]

Yeah, David Shapiro spoke about Polymorphic Apps a last year. Aka app that rewrites itself based on user desires. That has potential to remove software layer altogether. But I Was exploring this since then and we are not there yet... Currently models are not smart enough(12-13% change to solve one issue?) Take a long time to work trough things(Devin takes 4 hours to build simple website, you can have similar one using WIX in less time) And then there is cost. Matthew here spent two dollars not fixing the issue(and if he spent more there still was only 12% chance its gonna work?) In that sense for now something more like Zapier that controller by LLM that mashes together already working software seems like closer to reality. We need models to be 10x smarter, 100x faster, and probably 100x cheaper too for this to make any sense. I think biggest issue is reuse... They often write from scratch too.

[@quochung9999]

Check open interpreter please

[@robertheinrich2994]

@@quochung9999 so, something like this already exists. thank you

[@brianmi40]

Wait for your own custom OS...

[@covertassassin1885]

​@@brianmi40AIOS just came out 2 days ago

[@teamcoton]

Is that finally what we've been waiting for ?

[@niloymondal]

Yes, you can retire now.

[@smtkumar007]

I had used this and other differnt such coding agents after letting them run on my codebase for few hours , and spending 50$ on api charges already it came up with a correct solution , problem is the cost and time it take to complete such task, as a developer i can say until there doesn't comes a cheaper and a faster altearnative , fresher devs are safer. while senior devs still have few month/years until such modes are able to solve 90% of the times

[@manuelmontero1472]

Which one would you say has worked better? We had like devika the other day and that seemed okay

[@ddddevice]

I installed all your reviews.. !!! my old laptop ask me to die

[@Alf-Dee]

We now need a comparison video with usecases to better understand which open source agent project is better at what. Please do it 🙏

[@SuperJaury]

FYI: Aider is not using ctags anymore but tree-sitter, like Codeium does. Thanks for the video Matthew!

[@michamohe]

I'd like to see this in a multi-agent framework. I'd think it would improve the success rate exponentially

[@easypeasy2938]

just found your content.....I'm a 50 something trying to get back in the game....learning python...last language was Fortran...ouch. love your content and style...if you were in my shoes, what 10 pieces of your content would you watch in order to automate my life and existing businesses/side hustles?

[@Mythingyissososotinytho]

Everything you can get your hands on pops, and things have changed a lot , forget python past the basics, strap on and get a crash course in ML and Data Science, KZbin Edition 😅 struggle is real for us older cats eh!?Im 42 and can’t find my shoes until lunchtime and I thought a pomodoro was a hat. Whopper snappers best get them thar robotic thinking machines a goin else I ain’t gonna make it.

[@DaveEtchells]

Can you imagine the kind of coding assistants OpenAI has for their own internal use? It would make sense for them to have a team working on just that, because it would be a force multiplier for all their coders doing anything else. Given how fast they were moving and the fact that they haven’t released anything significant in a year (IMHO, 4 turbo was just incremental), we’re going to be blown away by whatever they release next. I’m sure some of it will bring a leap forward in code generation.

[@4.0.4]

I think you're overestimating how good these coding assistants are. Unless the "leaks"/speculation about Q* are true, it's still unable to assist much in wildly novel tasks, since they don't show up on the training data.

[@PaulDest]

Great to see a built-in 'cost limit' - for the first time, that brings the obfuscated topic of time and money a little bit out of the darkness. As all these Devins succeed in less than 15% according to the SWE benchmark, what happens with the other more than 85%? Do the Devins endlessly generate tokens and costs until interrupted by the supervising human (or a cost limit)? And even in the successful cases: how much time and tokens (=energy =costs even with local LLMs) does a task take for real? I'd really appreciate to see a comparison. Thank you, Matthew!

[@tsomerville1970]

can't wait to try it. for now, aider-chat is still my everyday "go to" - but i'm ready to try putting "issues" in my github similar to a prompt. i really want to understand pull requests better and was hoping you'd cover how the whole process works, and how we can use this practically. either i need to put crewai and use aider as a tool, or aider could add some cool agentic framework.

[@-UE-PR0]

Only problem with aider is that is uses openai api key

[@DesignMitho]

I can't believe it! I was looking for something like this for months!

[@MyCodingDiary]

Your videos always make my day. Keep shining!

[@sirishkumar-m5z]

SmythOS simplifies automated coding! Anyone trying to optimize their coding procedures has to have it. #SmythOS #Automation #AIAgent #Coding

[@blakemann2365]

Thank you for keep updating new “open devin” projects. Since there are now numerous projects, can you do a video to keep track and to compare those? It is very confusing.

[@VTC10English]

Among Devika, OpenDevin and SWE which one do you prefer?

[@RobertFitz-r8r]

Thank you Matthew for all your **great** videos. I really enjoy listening to them, esp. the ones on software development agents, like your Metagpt, Devika or this video. I suspect that 2024 will be the year of autonomous software development agents. Yes, they are not ready for prime time yet. But I am just waiting to see what happens when GPT-5 becomes available and is applied to these agent use cases.

[@marilynlucas5128]

Yeah this wiill enable me finish my autonomous game remastering studio. I am building an agent framework for remastering WOTLK using Azeroth Core!

[@FredPauling]

So much software to develop, so little time. This is cool.

[@tur_]

Good joh Matthew keep sharing these amazing projects

[@marilynlucas5128]

Giving an LLM its own ide is the future. This is why I predicted to several coding youtubers that in 5 years no human will be interacting with an ide.

[@genechristiansomoza4931]

I doubt that

[@patrickneal8131]

i'm in love with work and i have a request can you do a benchmark for AI agents that you already did, what you prefer?

[@christophejacquenet424]

BUT BUT !!?? DID YOU REVOKE THE OPEN API KEY AFTER THE VIDEO THO ??? (and thanks for your amazinf content !)

[@ernstmayer3868]

Would be great to see this working on a local repo with local LLM. Most code is not hosted on github.

[@BlueBearOne]

So...I'm a little confused. How is SWE Agent like Devin? Doesn't it just work on problems with software on GitHub? I'm looking for the best solution on my local computer to program code from scratch using an agentic setup. Not trying to fix stuff on GitHub. What am I missing here? Thank you.

[@rohitdas490]

This one is special, and we are ded

[@cjhmdm]

would love to see a video on creating a php/wordpress developer agency utilizing one of the many 'swarm' type tool out there.

[@Ayyouboss]

It seems like the docker image they provide is not Compatible with arm64 yet. So you need a x86 cpu to run this

[@todorp4056]

Great progress. Can it debug?

[@propeacemindfortress]

any of your local environments providing an openai like endpoint for your models should allow you to run it, just point it to the right local ip / port after setting up the endpoint

[@ScottzPlaylists]

👍 Nice video. SWE agent already works with Claude 2 (per the swe-bench link), please show how to make it use it❗ Because it uses GPT-4 API endpoint, can't you just use liteLLM to simulate the endpoint with any open source LM❓ .

[@propeacemindfortress]

sweet

[@yuvrajkukreja1248]

open source Ai-models are Awesome 🥳

[@Nairb932]

is SWE-Agent able to generate code based off of a prompt like Devin or does it only fix bugs in existing code?

[@MeinDeutschkurs]

This is also my question.

[@setevarmer]

Crafting this tech marvel wasn't the Herculean task it seemed; gathering materials for our Everest summit hotel was surprisingly straightforward. Yet, the real challenge lay not in the assembly but in the guest experience post-construction. Imagine embarking on the arduous climb to 29,032 feet, a journey where some falter, yet the sight of the hotel, seemingly close, revitalizes spirits. Alas, the unanticipated hurdle: a dress code, detailed in easily missed correspondence, requiring attire collection from Camp 2. The gatekeeper, steadfast and mute on the matter, turns away the uninformed, leaving them bewildered at the threshold.

[@01of13]

Hi Matthew. Thanks for your informative videos. Could you create one that shows how to use Swe-Agent on local repos?

[@RL.200]

Thanks!

[@marcfruchtman9473]

Very nice! Thanks for the video.

[@archerkee9761]

very nice thanks for the video Matt

[@rubiks.cube0008]

I've got this idea about ai. What if instead of texts or tokens or images the ai xan produce or generate signal or currents. In short, an ai that can directly control hardwares or control softwares directly. Imagine an ai that can control your entire pc like controlling the mouse.

[@ariaden]

Aww, I was hoping to see how the AI creates a pull request. I guess AI cannot improve itself yet, at least not without spending more than two bucks per improvement.

[@TomBielecki]

What is lightning? Got a link?

[@bradstudio]

Please let us know when you get a local LLM working with this.

[@kristianlavigne8270]

In a couple of year the best performers on the SWE benchmark will be at +50%

[@awesomecrome27]

You said about large context. How about Cursor IDE? it has a very long context and in gives you answers based on it.

[@angeldev96]

auto code rover just came out! please tutorial of that 🙏

[@GetzAI]

hey Matt, why are all of these only solving Github issue?

[@alansmithee419]

I think it's just because someone made a online package of Github repository tests specifically for the purpose of testing AIs, so that's what everyone uses because 1. It's the easiest way to test. 2. It's the easiest way to compare with others since everyone else is using it.

[@AIPulse118]

I'd like to see how this performs using claude sonnet 3.5

[@emporiumofthearcane]

i wonder if that miniconda error is something about the docker container not being able to connect to the outside world to download from that miniconda_url? kinda a wild guess. dunno man, good luck. thanks for the video!! looking forward to trying some of this locally later!

[@Cyberspider76]

You just need to change the docker file to use and/or make apple silicon image.

[@ScottSummerill]

I freaking can’t keep up. Feels like you either work for one of these companies or you’re left behind. Still better than the 90% who don’t even bother to follow. I guess.

[@alansmithee419]

If you work for one of the companies you'd probably be too focussed on AI news related directly to your work to pay attention to the full extent of its other uses and developments.

[@SupremeKingSovereign]

I am just waiting for someone to make an open-source language program. The language will be English and the libraries we call on will be small ai agents.

[@OlafStolle]

10:55 At this point you say you will use a local LLM. Do you also make a video about it?

[@GarethDoherty1985]

These all seem to be focused on python code. Do you know of any Agent tools on the horizon for .NET C# developers?

[@laukmedina]

Same

[@AI-OneNews]

My Questions was: Would it be possible that you guys add Ollama support? or is it already in progress? And someone of them answer that: This is already supported, see #5 . There was a small bug with Ollama that was fixed in #49. Lmk if you have further questions

[@AI-OneNews]

I tryed to post link here but got removed!

[@nikokroger]

Here: github.com/princeton-nlp/SWE-agent/issues/5

[@mrd6869]

Imagine this thing 3 months from now..We are heading into interesting territory

[@dtory]

I think we need more info on this agent ( another video )

[@HaseebHeaven]

Great project but still i would say need lots of improvements we need atleast 50% issues solving capabilities it too early for us to adapt this but great to see where its heading.

[@evanoslick4228]

Wonder if that window of code is not file based, but application syntax based.

[@betterlifeexe4378]

the non-codespace version doesn't have a ui 😮‍💨

[@VishalJangid1]

Hey you did not show you api keys this time 😅

[@myronkoch]

local model will be clutch

[@dennisestenson7820]

10:55, not much effort since text-generation-ui has an openai-compatible server that you can use with 3rd party apps just by changing URL from openai to your local server.

[@JT-Works]

Is there any ranking of these AI Agents somewhere? I tried Devinka and was pretty disappointed with it's performance.

[@HunterMayer]

Did we just automate open source?

[@mattmazurek]

aider logo used in this video (2:27) has nothing to do with aider tool you talk about :)

[@orangehatmusic225]

Man sure glad you all think using AI to make you money is nothing like slavery at all.

[@gokudomatic]

I tried devika with ollama mistral, and it couldn't even do the game of life official example. If I were you, I wouldn't bet too much on ollama for now.

[@ernesto.iglesias]

¿What si the SWE score of OpenDevin and Devika?

[@oryxchannel]

8:00 more meta. :-)

[@ahmetmutlu348]

this tools has to be integrated to GCC toolchain.. :P

[@mrdevolver7999]

You will love this secret dirty little trick every dev hates because AI steals their job... of fixing bugs while introducing a bunch of new ones...

[@simonsutton2946]

Can it only really deal with Parcel Tongue aka (python), what about full stack development pieces. Node js Typescript -React stuff. Or .Net

[@mhondoroinc1768]

how does it compare to Devika?

[@cfjlkfsjf]

Can these things act like an anti virus and scan for viruses and destroy them? I know jack about agents nor do I do anything that would require them.

[@kyber.octopus]

paying 2$ to solve a single bug doesn't sound great

[@myronkoch]

since it uses GPT4, can it help with Rust?

[@notsure221]

what if it executes your code and drops a db table..

[@watchdog163]

"OPENAI_API_BASE_URL: 'LM base URL here if using Local or alternative api Endpoint (optional)'"

[@effinballers2543]

why do i produce this error? CondaError: Run 'conda init' before 'conda activate'

[@AK-vx4dy]

So SWE-Box is an AI ?

[@jonmichaelgalindo]

FYI Docker doesn't work. I've struggled with it endlessly. Conda is top-tier, but Docker's whole premise of "let's add another abstraction layer and all hardware compat problems will vanish" is just nonsense. Docker only introduces a new layer of bugs. :-/

[@RobertFitz-r8r]

I also used the conda plus docker installation option and found one weird bug that is easy to fix. I ran it yesterday on a mac os system and had to correct the docker image name from "sweagent/swe-agent" to simply "swe-agent" to make it run correctly. You can use the docker tag option to easily fix that.

[@jonmichaelgalindo]

@@RobertFitz-r8r Probably means whoever made the docker image was on a mac with hardware config similar to yours. Doesn't help anybody.

[@Ravenofnorth]

Wait will this work for an code file not on GitHub?

[@Ravenofnorth]

Also can it generate code or only find bugs?

[@Liberty-scoots]

Can we use it unlike Devin?

[@jakebeau2790]

SWE is usually pronounced 'swee' -- stands for "software engineer"

[@OlstiMusic]

did we just gave AI the tool to create and fix itself? 😅

[@gavinkalikapersaud7225]

GPT WRAPPER?

[@itconsultanttech177]

can it work for java code base ?

[@jeffspaulding43]

I bet haiku could do as well for $0.02 :P

[@alansmithee419]

Why not use sonnet? Aren't they both free? Or is the API more expensive than the site itself?

[@executivelifehacks6747]

SWE will always be Society of Women Engineers to me.

[@48_subhambanerjee22]

Good. I think it's good and it will replace all CS jobs. And unemployment will be at a new peak. Only AI will prevail and humans will be forced to pick up odd jobs.

[@AetherTunes]

open source needs paid api key ...

[@TheNexusDirectory]

code is open source, not the api

[@hotjamsm07]

Great idea but too much trouble to install, IMHO.