ChatGPT "Code Interpreter" But 100% Open-Source (Open Interpreter Tutorial)

[@matthew_berman]

Should I do a follow up video on their new "OS Mode" which allows Open Interpreter to directly control your keyboard and mouse?

[@mahesh6307]

Definitely, 1000%

[@student7261]

yes. This is a cracking video. "OS Mode" will be even better. Keep up the good work !

[@pantsarmstrong7518]

One billion percent YES

[@aitruthbyace]

I'd love to see this!

[@f4r3l0]

Please add how to use Google gemini free api with open interpreter

[@OliNorwell]

The pacing of your videos are great, fast but easy to follow. I'm sure editing isn't easy but the results are great. Looking forward to trying this tonight on my machine.

[@matthew_berman]

Glad you like them!

[@Oli-l5m]

Love this video. AI is like employing a friendly idiot who doesn't know he's an idiot and is the perfect example of the reality of AI from a result perspective. I really hope this video becomes popular as its brilliant. Nobody else seems to publish stuff like this, please don't stop!

[@RodgerE2472]

Holy F*** this is amazing. Been using computers for over 40 years, I always dreamed of something like this. I started with an 8-bit Atari.

[@PatrickSteil]

Atari 1200XL was my first! With a cassette for a drive! I’d say we may some significant advances since then lol

[@kyrilgarcia]

Same. When i got my first computer (MSX) i thought i could ask it things and it would have "data inside". Turns out it didn't even have an operating system and i had to learn to code in Basic to use it and i didn't even know english 🤣 It's really amazing to see all of this finally happening

[@rjnash2610]

@@PatrickSteilYeah? well mine was a VIC20 w/tape drive so HA! (crap, all I did was expose I'm ollllddd... ffs)

[@gambiarran419]

ZX81 with the 4k ROM expansion pack! whoop whoop :) and Yes tech is certainly moving at pace now. @@PatrickSteil

[@DaveEtchells]

IMSAI 8080 with 8k of RAM, so there! 😉

[@mvasa2582]

Can you imagine - we went from command line to UI and now back to command line - (for efficiency)!

[@maidenlesstarnished8816]

That’s an interesting way to think about it! It totally kind of is a fancy command line

[@spicer41282]

Not so fast my friend... Killian and crew? Are building a GUI for it. Goal is to bring it neophytes... I prefer Simple . Command line will never go away. Way too efficient and effective!

[@ajarivas72]

@@spicer41282 What is GUI? I only use the mouse 🖱 to massage and relax my hands. No windows, no menus , no mouse. Just a terminal . The Macs 🖥 in verbose mode. What is a GUI?

[@homelessrobot]

some of us never left :P

[@Kazekoge101]

What is that project? You mean Ollama?@@spicer41282

[@josgraha]

yeah this has really blown up, thanks so much as always Matt. it would be cool if you did a follow up on the OS mode. you're demos are so clear and i really walk away with a fuller understanding of the core use cases and how to test them!

[@kyrilgarcia]

just installed it with mistral 7B running in LM studio, this is an absolutely amazing project that i'm gonna be using every day for a lot of stuff. thanks for the tutorial and review! 🙂

[@palmmutestudio5791]

how do you install code interpreter in LMstudio running an LLM?

[@AmanGusainGhost]

Hey, how was the performance?

[@kyrilgarcia]

@@AmanGusainGhost good enough for simple stuff but not as good as if you plug it into chatGPT. I have a "meh" computer with a 3060 and i can barely load a heavily quantized mixtral 8x7b, which is a lot better but really slow for me.

[@joetkeshub]

Exciting and great pedagogy. But it's really really dense. I had to watch your post partially on and on to get a decent working local. Thank you very much Matthew. There are several tutos on line but yours are particularly interesting. There's a lot of work behind I presume. ;p

[@warezit]

🎯 Key Takeaways for quick navigation: 00:00 🚀 *Introduction to Open Interpreter * - Overview of Open Interpreter, an open-source project similar to ChatGPT's code interpreter. - Can be run locally and powered by open-source models. - Significant improvements since the last review, including full computer control and developer-friendly features. 02:04 💻 *Demonstrating Open Interpreter's Capabilities * - Practical demonstration of controlling the computer using Open Interpreter. - Examples include folder management and image format conversion. - Highlights the ease and versatility of using Open Interpreter for various tasks. 04:07 🖼️ *Introducing Vision Feature and HTML Code Generation * - Introduction of the Vision feature in Open Interpreter. - Demonstrates creating HTML elements from images using the Vision feature. - Showcases Open Interpreter's ability to write and correct code dynamically. 07:09 🛠️ *Creating and Reusing Tools with Open Interpreter * - Explains how to create and save reusable tools in Open Interpreter. - Example of converting and saving a conversion tool script. - Emphasizes the convenience of reusing code for similar tasks in the future. 08:04 🌐 *Building Applications on Open Interpreter * - Guide on how to integrate Open Interpreter into applications. - Demonstrates using Open Interpreter within a Python environment. - Discusses potential issues and the self-correcting nature of the interpreter. 10:09 🌍 *Running Open Interpreter Locally with LM Studio * - Instructions for running Open Interpreter completely locally using LM Studio. - Discusses different model options and their impact on performance. - Highlights the flexibility and independence of using local models. 13:04 🧠 *Future Vision and Conclusion * - Discussion of the future vision for Open Interpreter and its potential impact. - Encourages experimentation and feedback on the Open Interpreter project. - Concludes with thoughts on the evolving interface of large language models with computers. Made with HARPA AI

[@rjnash2610]

pffft... show off

[@cloudnsec]

The potential for this with Cyber security is exciting!

[@endoflevelboss]

Awesome this channel is improving massively. No tie-dye hoodie. Good job 👍

[@LukasSmith827]

wow this is surreal to see! I wonder what the best coding model is and plug that into interpreter and see how it does

[@marcfruchtman9473]

Yet again, amazing progress. Thanks for this video.

[@johnsonjohnasson8115]

Loving the work Matthew. One thing. Any way you can type somewhere other than the bottom of the screen. Pausing puts the YT toolbar right where all the goodness is

[@joe_limon]

I can't wait for open source multi modal models

[@pipoviola]

You are a LEGEND already! Thank you very much

[@MarkusEicher70]

Thanks for this awesome into to Open Interpreter. This is truly the way to go. I just love it. Getting rid of all this proprietary bs with os, app stores and closed source models makes my day.

[@PierreMorelChannel]

For the lang="en" error I see a space between the les than symbols. What we see is < lang="en"> and should be

[@jamescarrico1233]

The end is near lol This is amazing

[@ITSupport-q1y]

Awesome work, thanks again Terence

[@WifeWantsAWizard]

(0:21) It can now fully-control your computer. That does sound perfectly safe, reasonable, and in no way liable in a court of law.

[@cesarsantos854]

If you ask ChatGPT to code a small script to just move your mouse, the censored thing refuses to do so. Thank god for open source!

[@ianrogers7543]

"interpreter -y" is how Skynet starts 🤣 I'm a big fan of GPTs in general, but this is a hugely resource-intensive replacement for a UI. What's the killer app for this? Command-lines are completely general, so a single assistant to replace all apps may be interesting, but, at this rate, future computers will need massively powerful GPUs just to run the command line...

[@brianmi40]

When GPT5 hits ALL THESE TOOLS get an instant upgrade... "What a time to be alive".

[@deathbysalt2382]

Great stuff. Would be interesting to see how Claude would do

[@legatuslabienus]

revolutionary and groundbreaking

[@jw8160]

Please use embedded captioning for your videos. The current method is very distracting and annoying to me, especially when the words cover things I want to see in the video. Having embedded captioning allows the viewer to turn on or off the captioning as necessary. Thank you. Edit: I see you are already using captioning, so please, please, please don't use more overlayed words outside of captioning. Thanks.

[@glowing_kitty]

Interesting video :) However: „I have a MacBook Pro M2 Ultra“. Congrats in that case! For being the first person on the planet for having that config ;) did you mean to say MacBook Pro with the M2 Pro, Max or a Mac with an M2 Ultra? And how many GB unified memory? (Since that is the bottle neck) Also, while I get the appeal of being able to control the whole computer by just chatting / talking to the computer, there will always be some kind of graphical user interface that displays data and allows us to work with them. But totally agree with AI / LLMs becoming the center point of our computers.

[@wholeness]

Sheesh!

[@thewelshninja]

Great stuff as always. Thanks

[@BruceWayne15325]

Thanks for sharing, this is great! I totally agree with you that AI driven architecture is the future of OS. In the not too distant future, when we have AGI, I can imagine that programs will not be monolithic entities, but rather more of a Linux style toolset approach, where each application just provides instructions for the AI on how to accomplish a given task. The AI could use those "applications" as needed to produce the desired result. The AI could create the interface on-the-fly to suit your needs, and your particular workflow. Eventually we could end up with the Star Trek holodeck, but in the near term we can approximate it on our screens.

[@krellin]

its awesome, now people will pay companies not only for running their apps but for writing their apps too :D soooo damn smart

[@MeinDeutschkurs]

MacBookPro M2 Ultra? :) That would be great! I think you mean Max.

[@matthew_berman]

Oops...you're right!

[@RobertWildling]

Wow! Self-correcting! Haven't seen that before (and yes, I do not use ChatGPT 4). I am still rather new to all this. I was wondering if you could make a tutorial about all those 7B, 11B, 13B... numbers. It seems there is a logic behind that, because it seems there aren't any 15B or 23B or 4.75B models out there. So far I haven't found a source that would help me understand. So I wonder if I could motivate you to cover the topic of models? 🙄

[@Alice_Fumo]

The numbers tell you how many parameters a model has which is indicative of the quality of their outputs and the amount of memory required to load it as well as the inference speed. For example, you'd expect a 14b model to perform better than a 7b model, but take twice the RAM and be twice as slow. The reason it is rare to see numbers which aren't 7 or 13 is that the vast majority of models are based off of Metas llama model, which just happened to be this size. To get a rough estimate of the RAM requirements of a model, you can divide by 2 if 4 bit or multiply by 2 in 16 bit precision. Thus the lowest possible amount of memory used by a 7b model in 4bit mode would be 3.5 gigabyte, but it's typically a little bit higher.

[@RobertWildling]

@@Alice_Fumo Wow! Thank you very much!

[@lancemarchetti8673]

Totally rocking!!

[@TomM-p3o]

They should implement function sharing so that the finished error free code can be uploaded to a database that could have functions pulled, also for further LLM training. PS What I suggested creates potential security vulnerabilities, so extra filters would need to be implemented. PS2 It's much easier for an LLM to analyze the code than to create it(for security issues)

[@NicksStuff]

It's so unbelievable I had to check if it was an April's fool joke.

[@Moukrea]

Feel it's more mature than Self-Operating Computer project, I would love to see how it handles the desktop in os mode for various tasks (i have the feeling it would control the target computer better if ran from an SSH session to prevent it to be confused with the terminal window)

[@kylea100]

You mention fully open-source on your own box. However, I don't think LM Studio is fully open source. Is it fully open source? Thanks for the great videos! They are always informative and useful.

[@ROKontotheFuture]

I love your videos as I've been using to install software you've recommended. I'm trying to figure out what you are running your code in as it doesn't look like my Terminal on iMac. What's the environment you run the code? Thanks for all the tips.

[@vincentnestler1805]

Thanks!

[@TomM-p3o]

I can't wait to use this with Mistral API

[@reinhardberger7083]

Thank you Matt for this again incredible video. I'm going for a Mac pro M2 Max myself, may I ask you which mem size would work for mistral/mixtral? (If Mixtral is possible at all locally, maybe quantized?)

[@brianWreaves]

Great share, cheers! 🏆

[@NutsAboutData]

04:28 Jesus, Mary, Joseph, and the chamels... this will revolutionize a lot

[@idris_pm]

Thanks for the great video. Does this sort or disrupt the robotic process automation now?

[@TokyoNeko8]

love this

[@OriginalRaveParty]

Cool ASF 😮

[@AdrienSales]

The local chapter is pretty cool : did you manage to run it on top of ollama ?

[@fxsurgeon1]

as exciting as it sounds, I'd hate to talk to my computer trying to explain what I need and bear the frustration of getting error after error. for simple stuff - yes; more complex tasks may still require some human intervention

[@RevMan001]

When you feel that pain in your chest and scream at your computer to delete all of your bookmarks and erase your browser history.... oh wait, just some gas. Awww man... I needed those bookmarks...

[@SODKGB]

FYI - I was not able to download Cody as of Jan 30, 2024 using a windows version of Visual Studio.

[@josersosa]

If you use "interpreter --config" you can chance the system prompt, models and other parameters

[@renierdelacruz4652]

Great video Matt, thanks very much. There is anyway to make it work on ollama, because ollama run faster than LM studio and has it´s own API ?

[@brianmi40]

You need one of the stand alone recorders that sits inline on the HDMI to your monitor (Amazon has tons)

[@golddiamond6923]

I was about to buy an Apple HomePod to fill that empty void, where this is all I was really looking for.

[@carmelusful]

NIce one here Thanks

[@NuncNuncNuncNunc]

Are locally downloaded models shared among different applications. If not shared, it seems to be a recipe for disaster orders of magnitude greater than NPM

[@tarikuwodajo1019]

Thank you

[@thomassynths]

No applications? That’s a completely unrealistic vision. Compute takes time and energy. We need specialized and preconceived software

[@tiagotiagot]

If even with GPT-4 it seems to keep making errors, seems a bit dangerous to let it run stuff on your computer; one f these days one of these errors might cause some serious damage to your files or something...

[@build.aiagents]

Thank you 🙏🏽

[@golddiamond6923]

I noticed a bug. I was trying to add an event for January 18th. and it still thinks it's 2023. So all this time I was telling OI it wasn't working, but it was. I'll need to tell OI the correct year from now on.

[@JohnDoe-sy6tt]

Nice!

[@sitedev]

How long before this is a standard component of your desktop OS? Next major release is my guess.

[@Viralvlogvideos]

6:07 ok my job is safe for now lol

[@kitrunner6596]

Your tutorials have been dope but every local AI that I have installed has been super wonky

[@DallasMcMillan]

Yes me too, even this video - all worked initially but then started giving password errors. This happens with almost every video of MB - I'm still very grateful because without his clear examples and explanations I'd never get close to getting these to run.

[@ruifigueiredo5695]

Hello Matthew, Do you think this will work with LM Studio to use code lama ?

[@elihusolano5993]

This is the true rabbit r1

[@kristoferkrus]

In VS Code, I suspect you need to activate the oi environment from the terminal that's running in VS Code and that you couldn't access the interpreter module because of that

[@GuildOfCalamity]

@13:18 I disagree, there will always be a need for applications, otherwise the AI would have to code and compile apps on the fly that are untested/unproven... which would be a disaster to the end user. With that said, it is getting closer, there just needs to be a tipping point in the culture.

[@OffTheBeatenPath_]

Slightly easier then installing PrivateGPT on Windows

[@oliverli9630]

waiting for the open source community to perfect the open source chatbots backend. we will detach from ClosedAI

[@chrisfortune9491]

It's promising, but frequently goes into death spirals and blows through 10,000 GPT tokens in a few seconds. GPT Pilot still wins, IMHO.

[@raminarizou7701]

Wow! Wow

[@lancemarchetti8673]

Vision is cool. Except it cannot work in ANSI encoded pages with invisible character bytes like in 0x81 0x8F, because there'll be no way for the OCR algorithm to detect those bytes.

[@lawrencium_Lr103]

So cool, Can you use it as an AutoGen agent?

[@emmanuelkolawole6720]

Where will you turn on the GPU, because I am having the same issue as you. Mixtral is not working on my m2 ultra. So the cpu is doing all the work. Where do i set GPU for the lm studio server

[@erdemguner]

Which recording tool you are usingv

[@Brenden-Harrison]

Why was it not able to see the flaw in the code or get it to work after multiple attempts? Is that not symptomatic of killer flaw that could be somehow a cap on its unique output and cognition ability?

[@retagainez]

11:20 Model was set to gpt-4, instead of your local mixtral model... could be a bug on the CLI output I suppose though.

[@darrenorange2982]

Is there a way to point it to another computer on the same network running LM Studio?

[@eyescreamcake]

I wish it were more like jupyter streamlit gradio and could display graphs and stuff

[@darrenorange2982]

Also it no longer seems to use LM Studio

[@tensiondriven]

But is it still Mac only? Once I can spin up an agent in a docker container that can run a gui, illl be stoked.

[@spicer41282]

Almost there! I asked it if OI can create its own memory capability. It said it can do it! However, it was almost completed until it crashed! in a specific sub routine app under PyQt. I believe it might be my fault? As I got greedy! :-/ I asked OI tons of UI features. My suggestion? When it asks you to run the script? y/n ⏸PAUSE! Think thru regarding running Oi's suggested script...What & How will OI suggested script affect your intended wholeCode? Will try again with zero Greed :-/ and a zero fast (food) auto code mental set!

[@riccoc.6507]

Do you need a monthly plan for gpt4 to do this? Im looking for something like this but not use chatgpt-4

[@malikrumi1206]

If we use openai with open interpreter, are all the contents of our computer that open interpreter touches going to end up in an openai db somewhere?

[@pstefan86]

Love this, though I worry a bit about the security credentials associated with these types of projects

[@JG27Korny]

I would worry to give LLM so much power in my pc.

[@MichaKurzewski]

'so of course python module issues' --- the story of my life

[@superfishintights901]

Have you ever had issues where it seems to struggle to execute code? It keeps timing out and never actually executing anything unless I ctrl+c. Arch + fish shell (tried it from within bash as well, no difference), and using conda / python3.11

[@destinyinskyx]

Giving sudo access to your machine to the chatbot. What if it executes maliciousness or some kind of hidden code.

[@rjnash2610]

wtf wow.... yet another leap achieved! someone pinch me!! HAAARRRD!

[@mbottambotta]

And vision using a local LLM?

[@eointolster]

Great project but very buggy, it cant find my d: and makes folers in the wrong place.

[@maz5232]

Is this work by google gemini api?

[@RiddleMaster-wi9yw]

For some reason when i run conda activate oi, it tells me to run conda init and it says no action taken but after that when i run conda activate oi again, it still asks me to run conda init

[@onbuttonup_1182]

Do you mean Mac *Studio* M2 Ultra ?

[@matthew_berman]

I meant Macbook Pro M2 Max sorry

[@onbuttonup_1182]

No worries! Great production you got going

[@Dietcheesey]

Great, now my grandmother can delete everything on my computer.

[@temesgenghidey566]

Is it possible to change the model to GPT 3.5 temporarily to avoid charges?