ChatGPT "Code Interpreter" But 100% Open-Source (Open Interpreter Tutorial)

This is my second video about Open Interpreter, with many new features and much more stability, the new Open Interpreter is amazing.
Update: Mixtral 7x8b wasn't working for me because I didn't have my GPU turned on, so it was running on pure CPU. 🤦‍♂️
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewberman.com
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew-berman-youtube
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
Open Interpreter - github.com/KillianLucas/open-...
Chapters:
0:00 - Introduction to Open Interpreter
0:35 - How to Install
4:02 - Using Vision
6:16 - Creating and Reusing Tools
7:43 - Writing Code on top of Open Interpreter
10:01 - Using OS Models

[@matthew_berman]

Should I do a follow up video on their new "OS Mode" which allows Open Interpreter to directly control your keyboard and mouse?

[@mahesh6307]

Definitely, 1000%

[@student7261]

yes. This is a cracking video. "OS Mode" will be even better. Keep up the good work !

[@pantsarmstrong7518]

One billion percent YES

[@aitruthbyace]

I'd love to see this!

[@f4r3l0]

Please add how to use Google gemini free api with open interpreter

[@OliNorwell]

The pacing of your videos are great, fast but easy to follow. I'm sure editing isn't easy but the results are great. Looking forward to trying this tonight on my machine.

[@matthew_berman]

Glad you like them!

[@josgraha]

yeah this has really blown up, thanks so much as always Matt. it would be cool if you did a follow up on the OS mode. you're demos are so clear and i really walk away with a fuller understanding of the core use cases and how to test them!

[@mvasa2582]

Can you imagine - we went from command line to UI and now back to command line - (for efficiency)!

[@maidenlesstarnished8816]

That’s an interesting way to think about it! It totally kind of is a fancy command line

[@spicer41282]

Not so fast my friend... Killian and crew? Are building a GUI for it. Goal is to bring it neophytes... I prefer Simple . Command line will never go away. Way too efficient and effective!

[@ajarivas72]

@@spicer41282 What is GUI? I only use the mouse 🖱 to massage and relax my hands. No windows, no menus , no mouse. Just a terminal . The Macs 🖥 in verbose mode. What is a GUI?

[@homelessrobot]

some of us never left :P

[@Kazekoge101]

What is that project? You mean Ollama?@@spicer41282

[@RodgerE2472]

Holy F*** this is amazing. Been using computers for over 40 years, I always dreamed of something like this. I started with an 8-bit Atari.

[@PatrickSteil]

Atari 1200XL was my first! With a cassette for a drive! I’d say we may some significant advances since then lol

[@kyrilgarcia]

Same. When i got my first computer (MSX) i thought i could ask it things and it would have "data inside". Turns out it didn't even have an operating system and i had to learn to code in Basic to use it and i didn't even know english 🤣 It's really amazing to see all of this finally happening

[@rjnash2610]

@@PatrickSteilYeah? well mine was a VIC20 w/tape drive so HA! (crap, all I did was expose I'm ollllddd... ffs)

[@gambiarran419]

ZX81 with the 4k ROM expansion pack! whoop whoop :) and Yes tech is certainly moving at pace now. @@PatrickSteil

[@user-pc2dz3bd3u]

Love this video. AI is like employing a friendly idiot who doesn't know he's an idiot and is the perfect example of the reality of AI from a result perspective. I really hope this video becomes popular as its brilliant. Nobody else seems to publish stuff like this, please don't stop!

[@marcfruchtman9473]

Yet again, amazing progress. Thanks for this video.

[@joetkeshub]

Exciting and great pedagogy. But it's really really dense. I had to watch your post partially on and on to get a decent working local. Thank you very much Matthew. There are several tutos on line but yours are particularly interesting. There's a lot of work behind I presume. ;p

[@pipoviola]

You are a LEGEND already! Thank you very much

[@user-wt7pq5qc2q]

Awesome work, thanks again Terence

[@thewelshninja]

Great stuff as always. Thanks

[@kyrilgarcia]

just installed it with mistral 7B running in LM studio, this is an absolutely amazing project that i'm gonna be using every day for a lot of stuff. thanks for the tutorial and review! 🙂

[@palmmutestudio5791]

how do you install code interpreter in LMstudio running an LLM?

[@amangusain7683]

Hey, how was the performance?

[@kyrilgarcia]

@@amangusain7683 good enough for simple stuff but not as good as if you plug it into chatGPT. I have a "meh" computer with a 3060 and i can barely load a heavily quantized mixtral 8x7b, which is a lot better but really slow for me.

[@lancemarchetti8673]

Totally rocking!!

[@endoflevelboss]

Awesome this channel is improving massively. No tie-dye hoodie. Good job 👍

[@jamescarrico1233]

The end is near lol This is amazing

[@deathbysalt2382]

Great stuff. Would be interesting to see how Claude would do

[@MarkusEicher70]

Thanks for this awesome into to Open Interpreter. This is truly the way to go. I just love it. Getting rid of all this proprietary bs with os, app stores and closed source models makes my day.

[@aimademerich]

Thank you 🙏🏽

[@TokyoNeko8]

love this

[@idris_pm]

Thanks for the great video. Does this sort or disrupt the robotic process automation now?

[@legatuslabienus]

revolutionary and groundbreaking

[@vincentnestler1805]

Thanks!

[@brianWreaves]

Great share, cheers! 🏆

[@LukasSmith827]

wow this is surreal to see! I wonder what the best coding model is and plug that into interpreter and see how it does

[@cloudnsec]

The potential for this with Cyber security is exciting!

[@ROKtotheFuture]

I love your videos as I've been using to install software you've recommended. I'm trying to figure out what you are running your code in as it doesn't look like my Terminal on iMac. What's the environment you run the code? Thanks for all the tips.

[@johnsonjohnasson8115]

Loving the work Matthew. One thing. Any way you can type somewhere other than the bottom of the screen. Pausing puts the YT toolbar right where all the goodness is

[@tarikuwodajo1019]

Thank you

[@reinhardberger7083]

Thank you Matt for this again incredible video. I'm going for a Mac pro M2 Max myself, may I ask you which mem size would work for mistral/mixtral? (If Mixtral is possible at all locally, maybe quantized?)

[@warezit]

🎯 Key Takeaways for quick navigation: 00:00 🚀 *Introduction to Open Interpreter * - Overview of Open Interpreter, an open-source project similar to ChatGPT's code interpreter. - Can be run locally and powered by open-source models. - Significant improvements since the last review, including full computer control and developer-friendly features. 02:04 💻 *Demonstrating Open Interpreter's Capabilities * - Practical demonstration of controlling the computer using Open Interpreter. - Examples include folder management and image format conversion. - Highlights the ease and versatility of using Open Interpreter for various tasks. 04:07 🖼️ *Introducing Vision Feature and HTML Code Generation * - Introduction of the Vision feature in Open Interpreter. - Demonstrates creating HTML elements from images using the Vision feature. - Showcases Open Interpreter's ability to write and correct code dynamically. 07:09 🛠️ *Creating and Reusing Tools with Open Interpreter * - Explains how to create and save reusable tools in Open Interpreter. - Example of converting and saving a conversion tool script. - Emphasizes the convenience of reusing code for similar tasks in the future. 08:04 🌐 *Building Applications on Open Interpreter * - Guide on how to integrate Open Interpreter into applications. - Demonstrates using Open Interpreter within a Python environment. - Discusses potential issues and the self-correcting nature of the interpreter. 10:09 🌍 *Running Open Interpreter Locally with LM Studio * - Instructions for running Open Interpreter completely locally using LM Studio. - Discusses different model options and their impact on performance. - Highlights the flexibility and independence of using local models. 13:04 🧠 *Future Vision and Conclusion * - Discussion of the future vision for Open Interpreter and its potential impact. - Encourages experimentation and feedback on the Open Interpreter project. - Concludes with thoughts on the evolving interface of large language models with computers. Made with HARPA AI

[@rjnash2610]

pffft... show off

[@joe_limon]

I can't wait for open source multi modal models

[@carmelusful]

NIce one here Thanks

[@JohnDoe-sy6tt]

Nice!

[@renierdelacruz4652]

Great video Matt, thanks very much. There is anyway to make it work on ollama, because ollama run faster than LM studio and has it´s own API ?

[@user-bd8jb7ln5g]

They should implement function sharing so that the finished error free code can be uploaded to a database that could have functions pulled, also for further LLM training. PS What I suggested creates potential security vulnerabilities, so extra filters would need to be implemented. PS2 It's much easier for an LLM to analyze the code than to create it(for security issues)

[@BruceWayne15325]

Thanks for sharing, this is great! I totally agree with you that AI driven architecture is the future of OS. In the not too distant future, when we have AGI, I can imagine that programs will not be monolithic entities, but rather more of a Linux style toolset approach, where each application just provides instructions for the AI on how to accomplish a given task. The AI could use those "applications" as needed to produce the desired result. The AI could create the interface on-the-fly to suit your needs, and your particular workflow. Eventually we could end up with the Star Trek holodeck, but in the near term we can approximate it on our screens.

[@user-bd8jb7ln5g]

I can't wait to use this with Mistral API

[@glowing_kitty]

Interesting video :) However: „I have a MacBook Pro M2 Ultra“. Congrats in that case! For being the first person on the planet for having that config ;) did you mean to say MacBook Pro with the M2 Pro, Max or a Mac with an M2 Ultra? And how many GB unified memory? (Since that is the bottle neck) Also, while I get the appeal of being able to control the whole computer by just chatting / talking to the computer, there will always be some kind of graphical user interface that displays data and allows us to work with them. But totally agree with AI / LLMs becoming the center point of our computers.

[@kylea100]

You mention fully open-source on your own box. However, I don't think LM Studio is fully open source. Is it fully open source? Thanks for the great videos! They are always informative and useful.

[@PierreMorelChannel]

For the lang="en" error I see a space between the les than symbols. What we see is < lang="en"> and should be

[@wholeness]

Sheesh!

[@brianmi40]

When GPT5 hits ALL THESE TOOLS get an instant upgrade... "What a time to be alive".

[@OriginalRaveParty]

Cool ASF 😮

[@ianrogers7543]

"interpreter -y" is how Skynet starts 🤣 I'm a big fan of GPTs in general, but this is a hugely resource-intensive replacement for a UI. What's the killer app for this? Command-lines are completely general, so a single assistant to replace all apps may be interesting, but, at this rate, future computers will need massively powerful GPUs just to run the command line...

[@raminarizou7701]

Wow! Wow

[@brianmi40]

You need one of the stand alone recorders that sits inline on the HDMI to your monitor (Amazon has tons)

[@Joe_Brig]

Any way to see the prompts that are going to the LLM?

[@gyvuneliu-uzkalbetojas]

I've tried it couple of weeks ago with gpt4. It was kinda cool, but at some point it was losing itself between a bunch of different files and when the project became more complex. Also the average cost of such coding in my case averaged about 10€/h

[@ruifigueiredo5695]

Hello Matthew, Do you think this will work with LM Studio to use code lama ?

[@sitedev]

How long before this is a standard component of your desktop OS? Next major release is my guess.

[@spicer41282]

Almost there! I asked it if OI can create its own memory capability. It said it can do it! However, it was almost completed until it crashed! in a specific sub routine app under PyQt. I believe it might be my fault? As I got greedy! :-/ I asked OI tons of UI features. My suggestion? When it asks you to run the script? y/n ⏸PAUSE! Think thru regarding running Oi's suggested script...What & How will OI suggested script affect your intended wholeCode? Will try again with zero Greed :-/ and a zero fast (food) auto code mental set!

[@JonathanYankovich]

But is it still Mac only? Once I can spin up an agent in a docker container that can run a gui, illl be stoked.

[@AdrienSales]

The local chapter is pretty cool : did you manage to run it on top of ollama ?

[@RobertWildling]

Wow! Self-correcting! Haven't seen that before (and yes, I do not use ChatGPT 4). I am still rather new to all this. I was wondering if you could make a tutorial about all those 7B, 11B, 13B... numbers. It seems there is a logic behind that, because it seems there aren't any 15B or 23B or 4.75B models out there. So far I haven't found a source that would help me understand. So I wonder if I could motivate you to cover the topic of models? 🙄

[@Alice_Fumo]

The numbers tell you how many parameters a model has which is indicative of the quality of their outputs and the amount of memory required to load it as well as the inference speed. For example, you'd expect a 14b model to perform better than a 7b model, but take twice the RAM and be twice as slow. The reason it is rare to see numbers which aren't 7 or 13 is that the vast majority of models are based off of Metas llama model, which just happened to be this size. To get a rough estimate of the RAM requirements of a model, you can divide by 2 if 4 bit or multiply by 2 in 16 bit precision. Thus the lowest possible amount of memory used by a 7b model in 4bit mode would be 3.5 gigabyte, but it's typically a little bit higher.

[@RobertWildling]

@@Alice_Fumo Wow! Thank you very much!

[@superfishintights901]

Have you ever had issues where it seems to struggle to execute code? It keeps timing out and never actually executing anything unless I ctrl+c. Arch + fish shell (tried it from within bash as well, no difference), and using conda / python3.11

[@emmanuelkolawole6720]

Where will you turn on the GPU, because I am having the same issue as you. Mixtral is not working on my m2 ultra. So the cpu is doing all the work. Where do i set GPU for the lm studio server

[@josersosa]

If you use "interpreter --config" you can chance the system prompt, models and other parameters

[@kumarapush2221]

How to make it take voice commands? If not direct, is there a way to do this through program? I have a small program that I built using python to listen to voice and convert voice to text & then reverse that will convert the text to speach. Now sure If I can integerate this to the Interpreter. It would be so cool.

[@NuncNuncNuncNunc]

Are locally downloaded models shared among different applications. If not shared, it seems to be a recipe for disaster orders of magnitude greater than NPM

[@kristoferkrus]

In VS Code, I suspect you need to activate the oi environment from the terminal that's running in VS Code and that you couldn't access the interpreter module because of that

[@lancemarchetti8673]

Vision is cool. Except it cannot work in ANSI encoded pages with invisible character bytes like in 0x81 0x8F, because there'll be no way for the OCR algorithm to detect those bytes.

[@lawrencium_Lr103]

So cool, Can you use it as an AutoGen agent?

[@tijendersingh5363]

Can you please make a video how to use quantised llava model with it

[@riccoc.6507]

Do you need a monthly plan for gpt4 to do this? Im looking for something like this but not use chatgpt-4

[@user-dl6ds6ij8x]

can we run interpreter with ollama please any help or pointers

[@elihusolano5993]

This is the true rabbit r1

[@darrenorange2982]

Is there a way to point it to another computer on the same network running LM Studio?

[@ps3301]

How about task weaver ? Do they function in the same way ?

[@RevMan001]

When you feel that pain in your chest and scream at your computer to delete all of your bookmarks and erase your browser history.... oh wait, just some gas. Awww man... I needed those bookmarks...

[@retagainez]

11:20 Model was set to gpt-4, instead of your local mixtral model... could be a bug on the CLI output I suppose though.

[@krellin]

its awesome, now people will pay companies not only for running their apps but for writing their apps too :D soooo damn smart

[@erdemguner]

Which recording tool you are usingv

[@mateuszpiechowiak2891]

Is there any tool like this but for controlling the web browser instead of the local computer?

[@mbottambotta]

And vision using a local LLM?

[@darrenorange2982]

Also it no longer seems to use LM Studio

[@malikrumi1206]

If we use openai with open interpreter, are all the contents of our computer that open interpreter touches going to end up in an openai db somewhere?

[@temesgenghidey566]

Is it possible to change the model to GPT 3.5 temporarily to avoid charges?

[@cesarsantos854]

If you ask ChatGPT to code a small script to just move your mouse, the censored thing refuses to do so. Thank god for open source!

[@TreeLuvBurdpu]

Can you install v2 in Termux? I'm having difficulty.

[@WifeWantsAWizard]

(0:21) It can now fully-control your computer. That does sound perfectly safe, reasonable, and in no way liable in a court of law.

[@pstefan86]

Love this, though I worry a bit about the security credentials associated with these types of projects

[@JG27Korny]

I would worry to give LLM so much power in my pc.

[@Brenden-Harrison]

Why was it not able to see the flaw in the code or get it to work after multiple attempts? Is that not symptomatic of killer flaw that could be somehow a cap on its unique output and cognition ability?

[@NicksStuff]

It's so unbelievable I had to check if it was an April's fool joke.

[@RiddleMaster-wi9yw]

For some reason when i run conda activate oi, it tells me to run conda init and it says no action taken but after that when i run conda activate oi again, it still asks me to run conda init

[@skyamaz5913]

Giving sudo access to your machine to the chatbot. What if it executes maliciousness or some kind of hidden code.

[@chadpogs7973]

Is it possible to install using vs code?

[@MeinDeutschkurs]

MacBookPro M2 Ultra? :) That would be great! I think you mean Max.

[@matthew_berman]

Oops...you're right!

[@attilavass6935]

Does it work if we instruct OI to create code using specific libraries, like langchain?

[@PatrickSteil]

Great question!

[@golddiamond6923]

I noticed a bug. I was trying to add an event for January 18th. and it still thinks it's 2023. So all this time I was telling OI it wasn't working, but it was. I'll need to tell OI the correct year from now on.

[@TiagoTiagoT]

If even with GPT-4 it seems to keep making errors, seems a bit dangerous to let it run stuff on your computer; one f these days one of these errors might cause some serious damage to your files or something...

[@NutsAboutData]

04:28 Jesus, Mary, Joseph, and the chamels... this will revolutionize a lot

[@SODKGB]

FYI - I was not able to download Cody as of Jan 30, 2024 using a windows version of Visual Studio.

[@MrGreenwolfie]

Any way to run it using text generation UI?

[@fxsurgeon1]

as exciting as it sounds, I'd hate to talk to my computer trying to explain what I need and bear the frustration of getting error after error. for simple stuff - yes; more complex tasks may still require some human intervention

[@golddiamond6923]

I was about to buy an Apple HomePod to fill that empty void, where this is all I was really looking for.

[@GuildOfCalamity]

@13:18 I disagree, there will always be a need for applications, otherwise the AI would have to code and compile apps on the fly that are untested/unproven... which would be a disaster to the end user. With that said, it is getting closer, there just needs to be a tipping point in the culture.

[@maz5232]

Is this work by google gemini api?

[@chrisshield3776]

what happened to deep seek?

[@jamesc2327]

Build with what language?

[@wow35master]

i have a question i am app developer sometimes i have small issues here and there, when i am continuing something in the team that i haven’t created instead of searching file to file manually to find the problem , can screenshot my problem and use the bot to find it and search the files if it will give also the solution thats a plus ?

[@Slav4o911]

Probably not, ChatGPT is extremely stupid for actual work... it's very good at "these demonstrations" but when you do actual work it's stupid like a brick. If your code is unique and not some general thing it will struggle or find "errors" where no errors are present also sometimes it would struggle to correct your wrong code or it's own wrong code.... or the "correction" will break the code even more. I have extremely bad experience, but some people say it's good. Maybe for some generic code... it's super good, but for unique code (not found in simple Google search) it's trash.... at least that's what I think.

[@wow35master]

@@Slav4o911 really thanks for you response! yes the code most of the times is unique, i just asked cuz recently we had a meeting at work and they allowed us to use ai stuff not on the server ofc just on our machines and i was wondering

[@Slav4o911]

@@wow35master You can still try it, it might work in your case. GPT4 seems to be more impressive to people who never interacted with it or in some specific cases, but overall I think it's not good enough it's not like a human it can't learn anything new and even things it already know (it can choose to forget for some reason). I was very impressed in the initial tests but then when I tried to implement it on my actual workflow it just doesn't work. It either slows me down or confuses me more in harder tasks, on easy tasks I can already do them faster. It's like a genious but sometimes it's brainless and it's like forgot how much is 1 + 1 (*figuratively speaking).