Local LangGraph Agents with Llama 3.1 + Ollama

[@RadThings]

Really liked this video going through Langgraph and using a local LLM. I like how you added Agent State and the use of the Semantic Router was very interesting. I'm definitely going to have to dig more into LangGraph. Thanks for the tutorial.

[@jamesbriggs]

glad it was helpful :)

[@alx8439]

I always considered asking LLM to spill out the python code directly with using whatever functions you want it to use works much more native, robust and reliable, than taking this detour with JSON-here-is-how-you-call-the-tool thing. The only small downside is to think in advance on how to run the LLM generated code in a safe sandbox, so it won't do any crazy shit. But there's a plethora of options for that

[@ward_jl]

Exactly what I was looking for. Learning so much from your LangGraph videos, thank you.

[@jamesbriggs]

glad to hear!

[@nikosterizakis]

Have piloted for a few months now. Simply not working… some tools get called, but generally loads of hallucinations and chains not completing, or progressing randomly. Same behaviour in Mac and Linux with Nvidia.

[@jamesbriggs]

llama 3.1 8b? It's a tiny model so hard to compare to OpenAI or others. However, I think forced function calls ollama.com/blog/tool-support (see under "Future improvements") will help a lot once released

[@nikosterizakis]

@@jamesbriggs Yes, Also tried with Mistral, Gemma2 and Phi3. I hope we see the progress based on the blogpost. Sorry to say but of the 3 agentic frameworks I have piloted for a while (this, Autogen, CrewAI), I find LangGraph the most 'hacky' to implement and inconsistent in behaviour. What you demonstrated here, and some other implementations I have seen and tried out, seem to break the patterns LangGraph have been trying to establish, in order to get 'things done'. Especially the 'routing' is an area they need to abstract better.. If I am to get 'deep down' in coding at that level, I may as well not use agents at all and go to 'functional / traditional' programming. I feel the framework is 1-2 years away from maturity, hopefully it will get there.

[@RealLexable]

I bet your development of a RAG would be one of the most exciting ones out there

[@jamesbriggs]

a RAG pipeline tutorial?

[@RealLexable]

@@jamesbriggs would be really awesome brother

[@jamesbriggs]

any of these work? Langgraph agent (incl. RAG tool) - kzbin.info/www/bejne/q6SynqqCj7OJq68 RAG + mistral AI - kzbin.info/www/bejne/f2HGZWNrgZxgb6M RAG w/ Llama 3 + Groq - kzbin.info/www/bejne/pJaQnaWjZdNjmMk Claude 3 RAG Chabot - kzbin.info/www/bejne/qJPdio2Cm7aVd68 Full LangChain RAG walkthrough (a little old): kzbin.info/www/bejne/gpnRdKavnKapsLs

[@RealLexable]

@@jamesbriggs seems so, but does it also include image data parsing or even video capturing or chating with own files and long memory store? Excuse my exploding wishbucket

[@nahiyanalamgir7056]

Try using uv instead of Poetry or Conda. It's noticeably faster and cleaner.

[@jamesbriggs]

I do now, it’s great!

[@juanargento3978]

Any benchmark between langgraph and semantic kernel for these type of agents? Thanks!

[@stevecoxiscool]

Learning a lot from the videos. How would I create an app which takes all my saved YT videos (I actual have this python code), and let me run a RAG query on them. Like "What video has langraph tutorials from Jame's". Also, it would be great to not only have the returned video urls, but also indexing into the video where such information was found (I have some code todo this, but I want an embedded video, not the whole webpage popping, up) if that makes sense. So I envision pinecone getting involved here. I save ALOT of YT videos in order to review them later in other words. Thanks !!!!

[@jamesbriggs]

yeah definitely a pinecone job - try this video, it's pretty outdated but should give you a feel for the flow of things kzbin.info/www/bejne/eZfYmYWMgpahqrM

[@mdipietro09]

why you didn't create the tools with the usual LangChain decorator "@tool" ?

[@jamesbriggs]

I ended up not using the langchain agent abstraction here, so no need to use the tool abstraction (as that is built for langchain's agent abstraction)

[@xspydazx]

yea yea yea bro ! but ! we know that the langgraph is not actually doing much ! your nodes are doing the work and the edges are just to connect the nodes theoretical : we will need them to create the edge matrix : which we cna use to check the graph from its start nodes to its proposed end odes : ( using shortest path) ... hence we can check all routes an paths : then all we need to execute a graph is execute the first node ! and return the final node!