Creating an AI Agent with LangGraph Llama 3 & Groq

This video picks up from the previous video and we convert the last Agent to be a LangGraph Agent and make it a bit more advanced. Still using Groq & Llama3 70B for the LLM
Colab: drp.li/X3hpZ (code)
🕵️ Interested in building LLM Agents? Fill out the form below
Building LLM Agents Form: drp.li/dIMes
👨‍💻Github:
github.com/samwit/langchain-t... (updated)
github.com/samwit/llm-tutorials
⏱️Time Stamps:
00:00 Intro
00:22 LangGraph Ecosystem
02:17 LangGraph Video
02:30 LangGraph Concepts
05:03 LangGraph Workflow
10:40 The Goal
12:17 Utility Function
12:46 Basic Chains
19:17 Tool Setup
19:39 Setting Up LangGraph State
20:50 Nodes
25:23 Conditional Edges
26:52 Build the Graph

[@wadejohnson4542]

Finally! A presentation on LangGraph that makes sense. Sign me up for any of your courses. I value your work.

[@an4aWebBasedSoftware]

great, solid base concepts! converting from colab to code is the perfect exercise to digest what you explained. Thank you !

[@marcintabedzki3578]

Fantastic intro to langgraph. It's awesome how well you explain this complex topics. Keep up the good work! Cannot wait more real life examples with rag.

[@kai_s1985]

Please make a video where rag is used! Most companies use their own data to answer questions rather than web search.

[@samwitteveenai]

I will I just realized it was going to be so long (time wise) for this one after explaining the LangGraph stuff. Probably next vid or early next week.

[@austinpatrick1871]

Yes I 100% agree

[@adithiyag4616]

Replace the tool like search with custom chain(which does rag) Congratulations, you successfully implemented RAG using agents

[@akhilsrivastava3371]

Superb intro. Thanks for making such amazing content

[@sayanosis]

Excellent video as always ❤

[@mr.daniish]

I have finally understood LangGraph!

[@teprox7690]

Thank you so much. I'm so happy that I found your content! ❤❤❤❤

[@samwitteveenai]

Thanks, glad it is helpful

[@el_arte]

In this contrived example, it doesn’t look like LangGraph adds a lot of value, but requires quite a bit of setup. I mean, a simple script with no opaque marshaling and a few conditionals could achieve the same thing.

[@yac7571]

Could you elaborate on this? Instead of using langgraph here, how would you write this application?

[@dylanlapierre9000]

I was thinking this same thing. I actually built a moderately complex rag chatbot with various conditionals as you mention calling what I guess you can think of as tools. It really makes me rethink what the term “AI AGENT” even means.

[@jerryyuan3958]

Great work

[@Munk-tt6tz]

I've learned so much from you, thank you so much!!!

[@SolidBuildersInc]

Thanks for your Birds Eye View of tying it all together. It has definitely relaxed my mind to have a structured approach that makes since. With this structure, I wonder if it can be built from a control sheet.............. which would introduce a DRY/RAD approach for AI.

[@maggiethedog]

Hi Sam. I love your content! I don't know if this is the proper way to report this, but in the colab notebook, the function def for 'route_to_rewrite' has the line 'research_info = state["research_info"]' which throws a runtime error, and the variable is not referenced in this module. Removing that line fixes the problem. Keep up the fantastic work Sam!

[@samwitteveenai]

thanks I will update it. I tend to write these pretty quick 😀

[@freedtmg16]

Dude keep it up. This is gold i only ask you build this stuff in a codebase like you might see in production. I find it really difficult to transfer code from ipynb to a vscode project, call it a mental block, and maybe I'm alone in feeling like this.

[@AndreiSheard]

I only have an iPad and an android phone so the fact he's doing all of this in Colab at the moment is a god- send haha A lot of other youtubers covering this stuff use VSCode. But, he's a great communicator so I definitely understand your request.

[@jdallain]

Really great examples for routing. It’s kinda hard to get that down from the LangGraph examples

[@riveww]

Awesome vid Sam! Question on the Schema Parsing and retrying that you do. It looks like moving on from a node requires the LLM to output a particular schema (like JSON with particular keys). Are there easy integration points with Pydantic/Instructor so we can be sure of our output schemas with retry logic while getting the benefits of LangGraph’s simple flow abstractions?

[@Alan0707]

It's great ! BTW, if there's a complex graph, it's hard to build the relations without a map

[@daniell.6463]

Great video! I really like how clean and professional your diagrams look. What tool are you using to create them? I've tried Graphviz before but the results just aren't as polished and engaging. Would love to know your process for making such appealing visuals. Keep up the awesome work!

[@samwitteveenai]

thanks for the kind words I am using Excalidraw for the diagrams. Super easy to use as well. Check it out.

[@daniell.6463]

@@samwitteveenai thank you!

[@theh1ve]

Would you say Sam it would be better to use LangGraph from the get go. It seems straight forward enough and it wpuld appear you can get up and running quickly. I just dont see the point going through CrewAI first then transition to LangGraph? Another great video too love your work!

[@el_arte]

⁠I slept over this and I now see a trend where people are obsessed with having any API interaction assisted or mediated by a LLM. It reminds me of the era of XML, when everyone wanted to use XML markup for everything, including network protocols. There’s a need to enable LLMs to interface with tools to extend their capabilities, but forcing natural language into every interaction seems a little weird. And, defining graphs to gate-keep reasoning flows seems brittle and limiting.

[@chainweaver]

This is fantastic. Can the steps in LangGraph be captured by Langsmith? is there a way you could show this. Debugging through Langsmith these steps would be awesome

[@andreyseas]

This is great. BTW, what do you use for a tool to design these flows to explain them?

[@samwitteveenai]

Thanks for the kind words I am using Excalidraw for the diagrams. Super easy to use as well. Check it out.

[@samansaadzadeh1833]

Thanks for video, I was getting key error when test for the other email when it needs to use the def route_to_rewrite(state). look like the research_info key is not required for def route_to_rewrite(state).

[@chainweaver]

Curious whether this could also work in a Chatbot experience. So in this example you had an email trigger the event but could this work if a user wanted more functionality but within a Chat experience. Maybe you might have some ideas on how that might actually work.

[@peterdecrem5872]

Thank you for the video. I think the graph allows to go write_draft_email->categorize_email->rewrite_email and rewrite_email assumes that state contains research_info (which I would not based on this path) or did i misread that? This seems to be a way to sidestep the lack of langchain tool support in groq (which I did not find - although there is tool support ) Thank you.

[@samwitteveenai]

It does a check after the categorize email if it needs research and then does research then a draft email and then another decision point for if it needs to rewrite. It is not using tools support as a function call, just using that Llama3 can handle JSON well and using that. You could also write some checking and retrying in there as well.

[@TzaraDuchamp]

Excellent explanation Sam, thanks. I have run the script with other models on Groq and got some errors. Have you tried to run it with models like "Mixtral-8x7b-32768", and "Gemma-7b-It"? Your last implementation with CrewAI seemed more robust, for me it ran with all models on Groq.

[@samwitteveenai]

This is super interesting as I didn't try this with those models, but I have done a bunch of stuff with Gemma and found it needed quite a bit of fine tuning to get it going with Agents. Thanks for testing it with the other models.

[@TzaraDuchamp]

Yes, your code runs without errors with Gemma, but that model and Llama 3 8b can't handle the agentic aspect with the given code. They report 'Agent stopped due to iteration limit or time limit.'. This adversely affects the BTC price inquiry mail response. Mixtral 8x7b runs well and handles the agentic aspect. Llama 3 70b can become a bit congested (waiting list, though haven't had it with the API) due to popularity, so it's a good option to have. I would be interested in you exploring Llama 3 8b's agentic prowess.

[@FinGloss]

Can you show how to integrate with Gmail and to run locally with our own data ? Also, how to train on our own data. THANKS

[@stephenzzz]

Sam, question if you don't mind. My wife wants to have her sales information content incorporated behind a chat/RAG to answer questions from her content. Which system out there do you think would work best, that is low code for a non-dev. Ideally next part will be to access this via a membership website.

[@samwitteveenai]

I am not up really on all the latest no code solutions, and privacy would be a big issue here. I do think Notion has done some really nice cool things with their adoption of RAG across all your databases etc

[@jarad4621]

Can we use this process somehow to search the web and do research? Can't find this anywhere

[@ps3301]

Instead of email, u should try to demonstrate langgraph using stock research agents!

[@samwitteveenai]

Have thought about doing this. Might take another look at it.

[@thedatascientist-lg4ls]

That's great, how about using an email from an account other than typing and passing the email prompt as it happens in the real world.

[@Salionca]

Dark mode, please. Thanks.

[@54peace]

Can I implement the same logic using JS instead??

[@samwitteveenai]

I think so but I haven't got around to trying LangGraph in JS.

[@nhtna4706]

What is groq’s role here ?

[@samwitteveenai]

It's serving the Llama3 70B model on their platform. Gives you much faster inference speeds

[@greendsnow]

Does anybody have unlimited groq api? Mine is not active.