Spring AI - Run Meta's LLaMA 2 Locally with Ollama 🦙 | Hands-on Guide | @Javatechie

#JavaTechie #SpringAI #Ollama #LLaMA2
👉 In this tutorial I will walk you through the steps to understand how to run LLM locally using ollama & spring ai , We will also work on a hands-on project with the popular LLM, Llama2
What You Will Learn:
👉 What is Spring AI?
👉 What is Ollama?
👉 What is LLaMA 2 ?
👉 hands-on project with the popular LLM, Llama2
Spring documentation : docs.spring.io/spring-ai/reference/api/index.html
🧨 Hurry-up & Register today itself!🧨
Devops for Developers course (Live class ) 🔥🔥:
javatechie.ongraphy.com/courses/Devops-for-Developers-64f1e07be4b0bbd6b56f7c05
COUPON CODE : NEW24
Spring boot microservice Premium course lunched with 70% off 🚀 🚀
COURSE LINK : Spring boot microservice course link :
javatechie.ongraphy.com/courses/Spring-Boot--Microservice-62c3e1f20cf25d97f28f9fa8
PROMO CODE : SPRING50
GitHub:
github.com/Java-Techie-jt/spring-ai-llama2
Blogs:
javatechie4u.medium.com/
Facebook:
groups/javatechie
Join this channel to get access to perks:
kzbin.infojoin
🔔 Guys, if you like this video, please do subscribe now and press the bell icon to not miss any update from Java Techie.
Disclaimer/Policy:
📄 Note: All uploaded content in this channel is mine and it's not copied from any community, you are free to use source code from the above-mentioned GitHub account.

[@dmode1535]

Thanks for posting this sir, I have learn a lot from you. Developers should be the highest paid group of any field because we do more learning than any field. We never stop learning, there is always something new to learn.

[@itdev7097]

"Developers should be highest paid group in any field".... exactly the opposite should happen.

[@Derrick-f8m]

man Busan you are awesome. you are the only guy on KZbin that innovates. you've been doing this for long time and never quit. No other teacher on this platform has continued to serve his dev community like this. Java will never die.

[@TheMrtest123]

Thanks for starting a series on AI. This is the need of the hour. Thanks for accepting our request for AI demos

[@_vku]

Thanks Basant sir, as a developer i like ur video to watch, whenever new leaning required i follow ur video.

[@praveenj3112]

Thank you for making this video and giving some basic info regarding the gen AI and spring AI

[@dhavasanth]

I appreciate your tireless efforts-thank you! We are eagerly awaiting more information on RAG and Vector Database.

[@gopisambasivarao5282]

Thanks Basant. Appreciate your efforts. God bless you!

[@grrlgd3835]

thanks for this JT. another great video. I'm going to take a look at the ETL framework for Data Engineering. look forward to more content

[@SouravDalal1981]

Great job. Will look forward for Vector database with AI integration

[@rajasekharkarampudi2669]

Great start to the Java AI world bro, Will be Waiting for the demos in this playlist on RAG applications using Vector DB, and Function calling, Take your time and plan a video on E2E real world projects, deploying it on a server.

[@SrkSalman416]

Very Good Video On Spring-AI. Your content(Videos) like Java Magnet, Its attract very fast.

[@abdus_samad890]

Loved it... Yes you can create more.

[@rabiulbiswas5908]

Thanks for the video. I hope more will come on Spring AI soon.

[@satyendrabhagbole3746]

Brilliant explanation

[@technoinshan]

Yes please create a detailed video with spring java and llm

[@kappaj01]

Great video. Can you do a RAG solution using the Weaviate Vector DB? I had one with AI 0.8 running, but it has changed so muched to 1.0...

[@Javatechie]

Sure I will give a try as currently I am learning it 😊

[@petersabraham7423]

Great video. Thank you for always making effort to update us with the latest technologies. I have one issue though, after implementing the generate rest api, my response takes too long, sometimes it takes up to 7minutes and i also noticed that you used GET request while llama "/api/chat" expects a POST request. Is there a particular reason you used a GET request? 2. Is it possible to train the llama model to recognize and provide responses based on the trained data?

[@ChandraSekhar-jm3sr]

Please share different model videos...they are very helpful

[@muralikrishna-qh6vg]

Thank you Basanth, can you please take us to Spring ai with google cloud (like vertexAiGemini) also vertextai with java

[@ProMaxGamingGuru]

Great start in AI world. I had question If I have an spring boot application and I want ans related to my application data only. how we can achieve that

[@Javatechie]

I am not sure will check and update you

[@srinivaschannel6230]

any course starting regarding Spring AI

[@Javatechie]

No buddy not planned any

[@MrSumeetshetty]

Thanks bro.. You are a good motivation for all developers to keep learning ❤

[@flutterdevfarm]

Sir, are you launching any latest course spring boot & microservices? The course which is listed on your site, is it Live?

[@attrayadas8067]

It's pre-recorded!

[@Javatechie]

It was recorded session of past live class currently I don't have any plan for new batch but If I have any plan in future then definitely I will update first in my KZbin channel for my audience 😀

[@mayankkumarshaw635]

Very nice ❤❤

[@Nilcha-2]

Is it possible to augment the model with local files (PDF, txt, docs) and then have llama scan through the files and answer any relevant questions in that file?

[@Javatechie]

Yes we can do that. I already did poc on it do let me know if you want video on this concept

[@Nilcha-2]

@@Javatechie Yes sir. I will greatly appreciate if you can do a video on that. Asking generic questions can also be done on free chatgpt and Gemini. The main use case is when business wants to provide their employees/customers with chatbot on their data. E.g HR policies, corporate plans, etc. A complex requirement is when a chatbot is required to parse database and summarize data, etc Currently we do that using Azure chatbot. But management do not like the idea of uploading confidential files. So if ollama can handle that locally and securely that will be the main use case.

[@psudhakarreddy6548]

Thank you

[@liqwis9598]

Hey basant nice video , can you please teach us how to use RAG functionality in locally running llms

[@Javatechie]

I will do this

[@liqwis9598]

@@Javatechie thank you as always 🙂

[@shrirangjoshi6568]

Please add the videos to a playlist.

[@Javatechie]

Sure buddy

[@rajkiran4572]

Thanks

[@Javatechie]

Thanks buddy 🙂👍

[@technoinshan]

hi getting below error when ever running that project java.lang.RuntimeException: [500] Internal Server Error - {"error":"model requires more system memory (8.4 GiB) than is available (7.5 GiB)"}

[@Javatechie]

Let me check this

[@vaibhavshetty3781]

getting this error while , Error: llama runner process has terminated: signal: killed

[@Javatechie]

At what step are you getting this error?

[@vaibhavshetty3781]

@@Javatechie while running docker exec -it ollama ollama run llama2

[@vaibhavshetty3781]

does it require any higher machine spec

[@Javatechie]

No specification required

[@abhishekkumar2020]

getting same error

[@moinakdasgupta3341]

Hi JT, I am running both ollama and my spring boot app using docker compose, but app is getting 500 response when hitting ollama api. This is fixed only when I manually run ollama run llama2 or ollama pull llama2 in the ollama container. Is there any way to automatically pull the model while starting from docker compose? I tried command: ["ollama", "pull", "llama2"] in docker compose file with no luck :(

[@Javatechie]

not sure buddy, I will check and update you

[@theniteshkumarjain]

Do we need tokens to generate the response? Also are these models for free?

[@Javatechie]

No tokens required yes these models are open-source

[@theniteshkumarjain]

@@Javatechie thanks

[@atulgoyal358]

I got below issue Error: model requires more system memory (8.4 GiB) than is available (3.9 GiB) docker exec -it ollama ollama run llama2

[@ashokpandit1367]

That's we are waiting for , let's give left and right to the Python people now 😅😅

[@RohitSharma-qb1vw]

Awesome video, Please make videos on remaining Models also.

[@2RAJ21]

I got below issue verifying sha256 digest writing manifest removing any unused layers success Error: model requires more system memory (8.4 GiB) than is available (2.9 GiB) after running -> docker exec -it ollama ollama run llama2 how to solve this ?? please help me..

[@atulgoyal358]

I got same issue ? did you got solution.

[@2RAJ21]

@@atulgoyal358 I did not touch after this issue. I think increase docker memory size.

[@atulgoyal358]

@@2RAJ21 Need to check how to increase docker memory size

[@2RAJ21]

@@atulgoyal358 no idea bro.. I am confused with ram or memory..

[@atulgoyal358]

@@2RAJ21 Need to configure .wslconfig file in %userprofile% increase RAM and processor then it will work..

[@koseavase]

Here comes Spring boot to challenge Python

[@rishiraj2548]

😎👍🏻💯🙏🏻

[@CenturionDobrius]

As usual, great job ❤ Please, if possible, work on your microphone voice quality recording

[@Javatechie]

Hello buddy thanks for your suggestion actually mic quality is good and things are echoing so will definitely try to improve it