Unlocking The Power Of AI: Creating Python Apps With Ollama!

Its amazing how easy the Python library for Ollama makes it to build AI into your apps. You can be up and running in minutes. This video gives you a nice overview of what is possible.
It seems I forgot to push the code to the repo before disappearing for the weekend to go camping. But it's there now. On a side note, Crescent Beach on the north coast of the Olympic Peninsula of Washington State is gorgeous....
Code for all the videos can be found at github.com/technovangelist/vi...
Be sure to sign up to my monthly newsletter at technovangelist.com/newsletter
And if interested in supporting me, sign up for my patreon at / technovangelist

[@matschbiem2082]

Great video to get started with the Python API, thanks a lot! :) Your way of explaining, presenting and so on is really great

[@randomdude12370]

I love the way you describe things. Great work!

[@utawmuddy5940]

you have no idea how good this info is for myself! wish I would have really seen this a while ago!

[@lundril]

I am just starting into the LLM and this video from you really helps. Thank You :-)

[@HashirAbdi]

Best non-intimidating and Clear Explanation of how to leverage the Python Ollama library. Tazeem (Respects)

[@tal7atal7a66]

this guy is professor , all my respects ❤

[@mwkoti]

Thanks for your work. Your videos are always spot-on

[@technovangelist]

Glad you like them!

[@NLPprompter]

OMG i love this one too. I really hope ollama will keep growing,

[@qbitsday3438]

Hi Mat thank you so much for all the great Videos , If you could do a video on how to Embed AI in to a chip (can be narrow AI) .

[@joetrades2472]

Id love to see in more detail how to set o llama in a cloud server as you did at the end. Also how to make it safe by stsblishing headers and certification

[@MuhammadAzhar-eq3fi]

Thank you Sir

[@stuartedwards3622]

Great stuff Matt. Was having a play with this today and have a question. In your script 7.py, you explain how to train the model with system / user / assistant messages and then get an output for a new user message (Amsterdam). How could we extend this to cover multiple messages (London, Brussels, Madrid etc) without having to reprompt the model with the system and assistant messages each time? I could only get a single output from each call to chat despite putting in several user messages.

[@nicolastecher577]

Fantastic video! Would you mind explaining how to export the app for others to use? I bet what makes the model work doesn’t get bundled when using py2app…..

[@vanderleigoncalves5184]

Very good

[@technovangelist]

Thanks.

[@cavalrycome]

Ollama also provides an OpenAI-compatible API, which is more convenient to code for because you can more easily switch between model servers that are compatible with that standard (OpenAI, Mistral, Groq, LM Studio, etc.). That version of the API is available under /v1 instead of /api.

[@technovangelist]

Using the OpenAI api should be an option of last resort. It’s there for compatibility but it’s always going to be less features than the native api. The OpenAI api was poorly designed, as stated by many of the OpenAI devs in various articles. They didn’t expect chatgpt to be useful to anyone. Many newer apis are more consistent and well thought out. Using the OpenAI api will definitely hold you back.

[@actellimQT]

Does this still apply? New openai API is out and response can't pass tools as far as I can reckon

[@userou-ig1ze]

How do I invest in you or your company, this guy is going places 😊

[@timtensor6994]

Hi Matt do you also have a video on tree summarization ? For example currently i am looking into reddit api where i can extract posts and posts description. The summarization of post description is relatively straightforward. Since it mostly comes within the context length . However with regards to get a good understanding of comments its a bit difficult . Is there a way to do that?

[@technovangelist]

interesting. That’s the way i just assumed would be how to summarize a long doc. Never knew someone put a name on it. dealing with comments is tough though. i assume reddit is like slack and discord where figuring out threads is hard.

[@timtensor6994]

@@technovangelist true but with some preprocessing, i could extract the comments , description , title and posts as pandas dataframe. The problem as mentioned is the comments, how to summarize them together while keeping context to ^basically lets say keep the llm in track , kind of like given this summary of the description please do recursive summary of comments .

[@akhilreddygogula8884]

Hii Matt, Great content, I'm really curious about ollama services deployed on k8s, scaling multiple instances on a local cluster for multi agent based application is a good use case. I mostly prefer working with AI locally with open source models as most of the community do. Hope you'll look at this. Thanks

[@technovangelist]

K8s is great for a lot of things but I don’t think this is one of them.

[@akhilreddygogula8884]

@@technovangelist thanks for reply, I got one more doubt, So Is it because, ollama doesn't support parallel inference(as of my knowledge) to serve multiple users or any other reason. Otherwise can you suggest a better inference serve that works with k8s??

[@technovangelist]

Its just a use case that isn't well suited for something like k8s. That’s great for tools that don't require a huge load. Ollama and similar tools will use 100% of a server. All the cpu and All the GPU. So k8s just takes some of that away. The Ollama team believes in kubernetes. We formed to create a tool for RBAC on K8S. But AI is not well suited to the platform.

[@jimlynch9390]

I'm a bit confused about base64 coding of image data. Since the context size is defaulted to 2k on ollama (I think I read that somewhere) how can you include image data that is encoded. I suspect even a trivial .png file will generate more than 2 k bytes. What am I missing? By the way, thanks for this video, it really helps.

[@chrisBruner]

I added an example to the github, but it's not been incorporated into the readme yet. Just do this type of thing. ollama.generate(model='llava', prompt='What is this image', images =['IMG_8798.JPG'])

[@dinoscheidt]

Just a way to transport the data (in a poor way). Imagine you only can write chat messages with another person, and want to send an image. You would come up with some weird string character based combination as well to describe each pixel in the image you originally want to send. On the other side, that string is than turned into what it should be: a binary bit map. But if your protocol can’t sent bit maps, you need to go a level higher and encode it in text, or emojis, or what ever. Welcome to the world of Python and the engineering craftsmanship level of „data scientists“.

[@cavalrycome]

First, context lengths are specified in terms of the number of tokens rather than bytes. Tokens are words or parts of words in the case of language models. Second, if you're using the models curated by Ollama, they will have whatever context length those models support. Llava 1.6 has a context length of 32K for example. When creating your own modelfile, you need to specify the context length as one of the parameters in it using the num_ctx parameter.

[@johnkoeck3702]

Can Ollama run locally with speech recognition and text to speech? I'm also curious if could be run locally on a raspberry pi 4 or 5 ?

[@technovangelist]

No but I have seen some folks have interesting addons that they mention in the discord

[@johnkoeck3702]

@technovangelist Thank you for replying so quickly.

[@jimlynch9390]

I don't see the intro-dev-python... in your videoprojects repo.

[@technovangelist]

Doh. Though I got it on before my internet went out.

[@henrijohnson7779]

Yep - I was searching also and could not find it up to now

[@arjunarihant7283]

yeah, its still not there

[@technovangelist]

yes it is. there is a link in the description

[@roborob2570]

Hi Matt, fascinating to learn that you can interface with ollama. Could you please point me to the 1.py and so on files ? I can not find the files on github and I tried :-). Much obliged, Rob

[@A_F_Innovate]

I love the pause and drink at the end I know it kills people.

[@Delchursing]

Would using Ollama allow us to make free llm on local machine?

[@technovangelist]

Yes, absolutely

[@Delchursing]

@@technovangelist wow, just getting into ai and a bit worried about tokens. Thank you.

[@FloodGold]

What no April Fools joke? Too bad you didn't flip the awkward end to the start, haha

[@armandCodes]

I'm the only one who can't find the .py files?

[@technovangelist]

go to the repo for all the video projects: github.com/technovangelist/videoprojects. they are sorted by date of the video and there is also the name of the topic.

[@technovangelist]

the location is also in the description