Automate AI Research with Crew.ai and Mozilla Llamafile

[@mattc3265]

Love being able to run this locally. Great vid 👏👏👏

[@alexanderheppening]

Great video, really helpful. THX

[@Nick__X]

awesome content dude!

[@raminderpalsingh123]

Great video. I am still getting charged for gpt-4o usage when my llamafile seems to be working fine? Is that expected? Also, how can I switch the code to point to gpt-4o-mini, to even further minimize my openAI API costs? THANKS A BUNCH!

[@SixTimesNine]

Very good vid - thanks!

[@vladimirmiletin4486]

Nice, thanks

[@RealLexable]

Do you know something about the data protection rules of serper for using their Google API?

[@MikeHeavers]

I think generally it’s not so much what tool you use as what you’re scraping and what the intended use is. Most sites have their own stipulations or licenses on what you can and can’t do with their site / content

[@procrastinatingrn3936]

So a free perplexity?

[@86dansu]

But you used an Open AI key and Serper key so this might not be running entirely local right ?

[@MikeHeavers]

The RAG bit - grabbing content from the internet, will require being online, correct. Inference is local, provided you are using Llamafile.

[@iliadobrevqqwqq]

Tks

[@dbreardon]

Now everyone needs a new computer and a Cuda graphics card which are massively expensive due to crypto mining and now AI servers. Local runs way too slow on my 3-4 year old laptop. Will have to see if new Intel and AMD chips with embedded NPU's provide any support for multiple LLMs run on local machines.

[@practical-ai-prototypes]

Fair point - performance on local is not as good as running on cloud infrastructure. Seems like "AI-enabled" PCs will be the new trend.

[@gymlin123]

but dont I still have to pay for tokens?

[@practical-ai-prototypes]

You don't have to pay for tokens if you run a local model

[@practical-ai-prototypes]

Update - I made an `app-input.py` script that allows you to create your own agent and task just by answering some questions in the command line.

[@JofnD]

Seems very useful! Is there an update video for this?

[@practical-ai-prototypes]

@@JofnD no - but same instructions, just run `python app-input.py` from the command line.

[@mattportnoyTLV]

Probably should've mentioned this is in beta. Don't you think?

[@themax2go]

you talk about running locally but using openai... sad

[@MikeHeavers]

Yeah this is a fair point. I should do another version where I use an open model

[@ryanbthiesant2307]

Not good. Does not show the problems of crew ai working with Ollama or any other lllm. Crewai persistently asks for open ai key. The good I discovered Mozilla lllm server thank you. Crew ai is really bad.

[@mandelafoggie9359]

So what is better than Crewai?

[@practical-ai-prototypes]

You don't have to use openAI or the API key - you can just remove it from the code. The Ollama file sample from the github repo shows you how to use Ollama. Note that Ollama is not an LLM - it just allows you to run LLMs locally.

[@practical-ai-prototypes]

@@mandelafoggie9359 You can try autogpt if you want - I found it harder to use.

[@ryanbthiesant2307]

​@@practical-ai-prototypes Thank you I will check that out, again. I think it still asks for a key, even a fake key. Even if you want to use ollama.

[@dougsilkstone]

Having a blast with your content Mike. We're doing real similar things on yt 🫡 thanks for the awesome vids

[@MikeHeavers]

Wow - ton of content, I'll check this out. Thanks for the comment.

[@dougsilkstone]

@@MikeHeavers ah just finding my feet. Its good fun!