Unlocking The Power Of AI: Creating Python Apps With Ollama!

Its amazing how easy the Python library for Ollama makes it to build AI into your apps. You can be up and running in minutes. This video gives you a nice overview of what is possible.
It seems I forgot to push the code to the repo before disappearing for the weekend to go camping. But it's there now. On a side note, Crescent Beach on the north coast of the Olympic Peninsula of Washington State is gorgeous....
Code for all the videos can be found at github.com/tec...
Be sure to sign up to my monthly newsletter at technovangelis...
And if interested in supporting me, sign up for my patreon at / technovangelist

[@tal7atal7a66]

this guy is professor , all my respects ❤

[@bonolio]

Straight up, I am not a programmer, but I know enough to hack together tools with python when I need to. I had some tasks that I wasn't to add some AI functionality to, but have never really coded against an LLM or run a local LLM. This video and your Up and running with Ollama video were brilliant. It was like a condensed info dump that gave just what I needed to go from zero to some functional solutions in just a few hours. I know I have so much more to learn, but in truth, but with just the few examples you have provided, the possibilies are endless. I am subscribed and will be trawling through the rest of your content as a priority.

[@KOSMIKFEADRECORDS]

please put me on to the video... what real world usecase have you now found?

[@HashirAbdi]

Best non-intimidating and Clear Explanation of how to leverage the Python Ollama library. Tazeem (Respects)

[@randomdude12370]

I love the way you describe things. Great work!

[@matschbiem2082]

Great video to get started with the Python API, thanks a lot! :) Your way of explaining, presenting and so on is really great

[@utawmuddy5940]

you have no idea how good this info is for myself! wish I would have really seen this a while ago!

[@lundril]

I am just starting into the LLM and this video from you really helps. Thank You :-)

[@GonzoGonschi]

finally someone with explaining skills :)

[@mwkoti]

Thanks for your work. Your videos are always spot-on

[@technovangelist]

Glad you like them!

[@TorsTechTalk]

wonderful video, subbed! Thank you

[@NLPprompter]

OMG i love this one too. I really hope ollama will keep growing,

[@cavalrycome]

Ollama also provides an OpenAI-compatible API, which is more convenient to code for because you can more easily switch between model servers that are compatible with that standard (OpenAI, Mistral, Groq, LM Studio, etc.). That version of the API is available under /v1 instead of /api.

[@technovangelist]

Using the OpenAI api should be an option of last resort. It’s there for compatibility but it’s always going to be less features than the native api. The OpenAI api was poorly designed, as stated by many of the OpenAI devs in various articles. They didn’t expect chatgpt to be useful to anyone. Many newer apis are more consistent and well thought out. Using the OpenAI api will definitely hold you back.

[@actellimQT]

Does this still apply? New openai API is out and response can't pass tools as far as I can reckon

[@joetrades2472]

Id love to see in more detail how to set o llama in a cloud server as you did at the end. Also how to make it safe by stsblishing headers and certification

[@qbitsday3438]

Hi Mat thank you so much for all the great Videos , If you could do a video on how to Embed AI in to a chip (can be narrow AI) .

[@bruhaspati560]

In chat completion is there something like chat request with a pdf file?

[@stuartedwards3622]

Great stuff Matt. Was having a play with this today and have a question. In your script 7.py, you explain how to train the model with system / user / assistant messages and then get an output for a new user message (Amsterdam). How could we extend this to cover multiple messages (London, Brussels, Madrid etc) without having to reprompt the model with the system and assistant messages each time? I could only get a single output from each call to chat despite putting in several user messages.

[@nicolastecher577]

Fantastic video! Would you mind explaining how to export the app for others to use? I bet what makes the model work doesn’t get bundled when using py2app…..

[@jimlynch9390]

I don't see the intro-dev-python... in your videoprojects repo.

[@technovangelist]

Doh. Though I got it on before my internet went out.

[@henrijohnson7779]

Yep - I was searching also and could not find it up to now

[@arjunarihant7283]

yeah, its still not there

[@technovangelist]

yes it is. there is a link in the description

[@vanderleigoncalves5184]

Very good

[@technovangelist]

Thanks.

[@MuhammadAzhar-eq3fi]

Thank you Sir

[@userou-ig1ze]

How do I invest in you or your company, this guy is going places 😊

[@timtensor6994]

Hi Matt do you also have a video on tree summarization ? For example currently i am looking into reddit api where i can extract posts and posts description. The summarization of post description is relatively straightforward. Since it mostly comes within the context length . However with regards to get a good understanding of comments its a bit difficult . Is there a way to do that?

[@technovangelist]

interesting. That’s the way i just assumed would be how to summarize a long doc. Never knew someone put a name on it. dealing with comments is tough though. i assume reddit is like slack and discord where figuring out threads is hard.

[@timtensor6994]

@@technovangelist true but with some preprocessing, i could extract the comments , description , title and posts as pandas dataframe. The problem as mentioned is the comments, how to summarize them together while keeping context to ^basically lets say keep the llm in track , kind of like given this summary of the description please do recursive summary of comments .

[@roborob2570]

Hi Matt, fascinating to learn that you can interface with ollama. Could you please point me to the 1.py and so on files ? I can not find the files on github and I tried :-). Much obliged, Rob

[@akhilreddygogula8884]

Hii Matt, Great content, I'm really curious about ollama services deployed on k8s, scaling multiple instances on a local cluster for multi agent based application is a good use case. I mostly prefer working with AI locally with open source models as most of the community do. Hope you'll look at this. Thanks

[@technovangelist]

K8s is great for a lot of things but I don’t think this is one of them.

[@akhilreddygogula8884]

@@technovangelist thanks for reply, I got one more doubt, So Is it because, ollama doesn't support parallel inference(as of my knowledge) to serve multiple users or any other reason. Otherwise can you suggest a better inference serve that works with k8s??

[@technovangelist]

Its just a use case that isn't well suited for something like k8s. That’s great for tools that don't require a huge load. Ollama and similar tools will use 100% of a server. All the cpu and All the GPU. So k8s just takes some of that away. The Ollama team believes in kubernetes. We formed to create a tool for RBAC on K8S. But AI is not well suited to the platform.

[@johnkoeck3702]

Can Ollama run locally with speech recognition and text to speech? I'm also curious if could be run locally on a raspberry pi 4 or 5 ?

[@technovangelist]

No but I have seen some folks have interesting addons that they mention in the discord

[@johnkoeck3702]

@technovangelist Thank you for replying so quickly.

[@SailAway33]

I love the pause and drink at the end I know it kills people.

[@jimlynch9390]

I'm a bit confused about base64 coding of image data. Since the context size is defaulted to 2k on ollama (I think I read that somewhere) how can you include image data that is encoded. I suspect even a trivial .png file will generate more than 2 k bytes. What am I missing? By the way, thanks for this video, it really helps.

[@chrisBruner]

I added an example to the github, but it's not been incorporated into the readme yet. Just do this type of thing. ollama.generate(model='llava', prompt='What is this image', images =['IMG_8798.JPG'])

[@dinoscheidt]

Just a way to transport the data (in a poor way). Imagine you only can write chat messages with another person, and want to send an image. You would come up with some weird string character based combination as well to describe each pixel in the image you originally want to send. On the other side, that string is than turned into what it should be: a binary bit map. But if your protocol can’t sent bit maps, you need to go a level higher and encode it in text, or emojis, or what ever. Welcome to the world of Python and the engineering craftsmanship level of „data scientists“.

[@cavalrycome]

First, context lengths are specified in terms of the number of tokens rather than bytes. Tokens are words or parts of words in the case of language models. Second, if you're using the models curated by Ollama, they will have whatever context length those models support. Llava 1.6 has a context length of 32K for example. When creating your own modelfile, you need to specify the context length as one of the parameters in it using the num_ctx parameter.

[@Delchursing]

Would using Ollama allow us to make free llm on local machine?

[@technovangelist]

Yes, absolutely

[@Delchursing]

@@technovangelist wow, just getting into ai and a bit worried about tokens. Thank you.

[@FloodGold]

What no April Fools joke? Too bad you didn't flip the awkward end to the start, haha

[@armandCodes]

I'm the only one who can't find the .py files?

[@technovangelist]

go to the repo for all the video projects: github.com/technovangelist/videoprojects. they are sorted by date of the video and there is also the name of the topic.

[@technovangelist]

the location is also in the description