Creating an AI Agent with LangGraph Llama 3 & Groq

[@bonadio60]

Langgraph should add this video at their site. Its a great explanation much simpler and to the point. Thanks so much.

[@kakashisensie100]

i think they tweeted link to the video

[@wadejohnson4542]

Finally! A presentation on LangGraph that makes sense. Sign me up for any of your courses. I value your work.

[@an4aWebBasedSoftware]

great, solid base concepts! converting from colab to code is the perfect exercise to digest what you explained. Thank you !

[@out_and_about08]

I absolutely love your videos and pace and depth of it ! Thanks a lot :)

[@samwitteveenai]

Thanks for the comment and kind words.

[@maggiethedog]

Hi Sam. I love your content! I don't know if this is the proper way to report this, but in the colab notebook, the function def for 'route_to_rewrite' has the line 'research_info = state["research_info"]' which throws a runtime error, and the variable is not referenced in this module. Removing that line fixes the problem. Keep up the fantastic work Sam!

[@samwitteveenai]

thanks I will update it. I tend to write these pretty quick 😀

[@adnanbarwaniwala8686]

You are the goat!! Please never stop making such videos!!

[@hxxzxtf]

🎯 Key points for quick navigation: 00:30 *🤔 LangGraph is a middle ground between LangChain and fully autonomous agents, providing structure and state to create agents that can be monitored and controlled.* 01:26 *🔒 CrewAI is suitable for testing ideas and prototyping, but not for production due to lack of control over the next steps.* 02:25 *📊 LangGraph consists of chains, nodes, edges, and conditional edges that connect nodes together to form a graph.* 03:51 *💡 The state graph maintains variables that get passed around from node to conditional edge to node as it goes through the flow.* 05:21 *⚖️ Conditional edges decide which node to go to next based on conditions such as research or rewriting an email.* 06:23 *📝 The state graph saves variables such as email categories and research results for use in subsequent nodes.* 07:25 *🔍 Conditional edges can be used for more advanced tasks such as checking brand standards or competitor mentions.* 09:51 *📝 LangGraph provides more control over the AI agent's flow and decision-making process compared to CrewAI.* 10:21 *💻 The code imports necessary libraries, including langchain.groq for Llama 3, and sets up the goal of categorizing an email and deciding whether to do research or not.* 11:24 *🔁 The agent will have a recursive improvement step to check if a draft reply needs rewriting.* 11:56 *📊 The chat model is set up using Llama 3 on Groq, with chat prompt templates and output passes for JSON and string outputs.* 12:29 *👉 Chains in LangGraph are used to categorize emails, decide on research, generate keywords, write draft emails, and check for rewrites.* 13:01 *🔄 Each chain has its own specific function in the agent's workflow.* 14:28 *💡 Testing each chain individually allows for debugging and refinement of the agent's performance.* 18:57 *📝 The model can be run on-premises, and the rewrite prompt is used for rewriting emails.* 19:24 *📊 The LangGraph part of the agent consists of nodes, edges (normal and conditional), and state.* 20:26 *💡 The state is used to pass variables around in the agent, allowing for tracking of variables like initial email, email category, draft email, and research info.* 20:55 *👉 The nodes in the LangGraph include categorized email node, research info node, draft email writer node, and others.* 21:25 *🔁 Print statements are used to help debug the agent's behavior.* 22:28 *🔎 Search functionality is used to retrieve documents based on keywords from an initial search query.* 23:28 *📝 The state dictionary is updated with new information as each node is called.* [24:59](outu [28:01] 📝 The standard edge in a graph is a hardwired edge that controls the flow of nodes. [28:34] 🔀 Conditional edges make decisions based on input and can change the flow of nodes. [29:06] 💡 Conditional edges can be used to decide which node to go to next, and can be set up to make decisions based on input. [30:05] 📊 The state printer node is used to see the final variables and outputs of the graph. [31:01] 💻 Once the workflow is set up, it can be compiled as an app and used with inputs. [32:38] 📝 The graph can be customized and reused with different inputs and prompts. [34:07] 🔧 Loops can be added to the graph, but care must be taken to avoid getting stuck in an infinite loop. Made with HARPA AI

[@el_arte]

In this contrived example, it doesn’t look like LangGraph adds a lot of value, but requires quite a bit of setup. I mean, a simple script with no opaque marshaling and a few conditionals could achieve the same thing.

[@yac7571]

Could you elaborate on this? Instead of using langgraph here, how would you write this application?

[@dylanlapierre9000]

I was thinking this same thing. I actually built a moderately complex rag chatbot with various conditionals as you mention calling what I guess you can think of as tools. It really makes me rethink what the term “AI AGENT” even means.

[@anhhct]

think of it as a way that you can have your logic modified in the future without much changes in the code. Like all the steps and edges can stay the same, just the way you wire them together changes. Of course that also can be achieved without LangGraph as well though. Just like you can even build your llm application without LangChain. The AI and Python community seem to prefer ways of doing things with abstractions and prebuilt bells and whistles, just like the way the language is set up, clear and easy to read.

[@marcintabedzki3578]

Fantastic intro to langgraph. It's awesome how well you explain this complex topics. Keep up the good work! Cannot wait more real life examples with rag.

[@databasemadness]

I have finally understood LangGraph!

[@SolidBuildersInc]

Thanks for your Birds Eye View of tying it all together. It has definitely relaxed my mind to have a structured approach that makes since. With this structure, I wonder if it can be built from a control sheet.............. which would introduce a DRY/RAD approach for AI.

[@anhhct]

Thank you! Your style of explanation is very clear.

[@kai_s1985]

Please make a video where rag is used! Most companies use their own data to answer questions rather than web search.

[@samwitteveenai]

I will I just realized it was going to be so long (time wise) for this one after explaining the LangGraph stuff. Probably next vid or early next week.

[@austinpatrick1871]

Yes I 100% agree

[@adithiyag4616]

Replace the tool like search with custom chain(which does rag) Congratulations, you successfully implemented RAG using agents

[@emko6892]

Connected to a database

[@wernerpjjacob6499]

Very good Didatic !!!! Thank You So Much!!!!

[@diegocalderon3221]

CrewAI is essentially a LangChain wrapper (and IMO not a good one). This is actually a great show case of what customization you can do. If you need flexibility for your Agents, then LangGraph is the way to go.

[@freedtmg16]

Dude keep it up. This is gold i only ask you build this stuff in a codebase like you might see in production. I find it really difficult to transfer code from ipynb to a vscode project, call it a mental block, and maybe I'm alone in feeling like this.

[@AndreiSheard]

I only have an iPad and an android phone so the fact he's doing all of this in Colab at the moment is a god- send haha A lot of other youtubers covering this stuff use VSCode. But, he's a great communicator so I definitely understand your request.

[@jzam5426]

Thank you! It seems like the graph is built based of pre-defined chains and not an Agent who makes decision to independently call tools (kind of like crewai/autogen). Can you make a video where the agent makes decisions on a series of steps of what tools to use, through tool calling please?

[@akhilsrivastava3371]

Superb intro. Thanks for making such amazing content

[@daniell.6463]

Great video! I really like how clean and professional your diagrams look. What tool are you using to create them? I've tried Graphviz before but the results just aren't as polished and engaging. Would love to know your process for making such appealing visuals. Keep up the awesome work!

[@samwitteveenai]

thanks for the kind words I am using Excalidraw for the diagrams. Super easy to use as well. Check it out.

[@daniell.6463]

@@samwitteveenai thank you!

[@PhattharaphonRomphet-kg3oj]

Thank you about the knowledge very good explanation

[@teprox7690]

Thank you so much. I'm so happy that I found your content! ❤❤❤❤

[@samwitteveenai]

Thanks, glad it is helpful

[@HomunMage]

Thanks for sharing. I have intrest in LangGraph with llama 3 on local such use ollama

[@sayanosis]

Excellent video as always ❤

[@Munk-tt6tz]

I've learned so much from you, thank you so much!!!

[@SirajFlorida]

This is just the descripting of an agent based Turing Machine. Definitely cool. "

[@jdallain]

Really great examples for routing. It’s kinda hard to get that down from the LangGraph examples

[@nogool111]

Can you make a video that uses Lang Graph to create a cycle that repeatedly calls an LLM in a loop to determine the next action? I think what Lang Graph is designed for. In this example, we can definitely use the Lang Chain to achieve the same result because it linear. Anyway, your video is more easier to understand than the Lang Graph document. Thank you.

[@riveww]

Awesome vid Sam! Question on the Schema Parsing and retrying that you do. It looks like moving on from a node requires the LLM to output a particular schema (like JSON with particular keys). Are there easy integration points with Pydantic/Instructor so we can be sure of our output schemas with retry logic while getting the benefits of LangGraph’s simple flow abstractions?

[@chainweaver]

This is fantastic. Can the steps in LangGraph be captured by Langsmith? is there a way you could show this. Debugging through Langsmith these steps would be awesome

[@Alan0707]

It's great ! BTW, if there's a complex graph, it's hard to build the relations without a map

[@jerryyuan3958]

Great work

[@andreyseas]

This is great. BTW, what do you use for a tool to design these flows to explain them?

[@samwitteveenai]

Thanks for the kind words I am using Excalidraw for the diagrams. Super easy to use as well. Check it out.

[@theh1ve]

Would you say Sam it would be better to use LangGraph from the get go. It seems straight forward enough and it wpuld appear you can get up and running quickly. I just dont see the point going through CrewAI first then transition to LangGraph? Another great video too love your work!

[@el_arte]

⁠I slept over this and I now see a trend where people are obsessed with having any API interaction assisted or mediated by a LLM. It reminds me of the era of XML, when everyone wanted to use XML markup for everything, including network protocols. There’s a need to enable LLMs to interface with tools to extend their capabilities, but forcing natural language into every interaction seems a little weird. And, defining graphs to gate-keep reasoning flows seems brittle and limiting.

[@chainweaver]

Curious whether this could also work in a Chatbot experience. So in this example you had an email trigger the event but could this work if a user wanted more functionality but within a Chat experience. Maybe you might have some ideas on how that might actually work.

[@peterdecrem5872]

Thank you for the video. I think the graph allows to go write_draft_email->categorize_email->rewrite_email and rewrite_email assumes that state contains research_info (which I would not based on this path) or did i misread that? This seems to be a way to sidestep the lack of langchain tool support in groq (which I did not find - although there is tool support ) Thank you.

[@samwitteveenai]

It does a check after the categorize email if it needs research and then does research then a draft email and then another decision point for if it needs to rewrite. It is not using tools support as a function call, just using that Llama3 can handle JSON well and using that. You could also write some checking and retrying in there as well.

[@TzaraDuchamp]

Excellent explanation Sam, thanks. I have run the script with other models on Groq and got some errors. Have you tried to run it with models like "Mixtral-8x7b-32768", and "Gemma-7b-It"? Your last implementation with CrewAI seemed more robust, for me it ran with all models on Groq.

[@samwitteveenai]

This is super interesting as I didn't try this with those models, but I have done a bunch of stuff with Gemma and found it needed quite a bit of fine tuning to get it going with Agents. Thanks for testing it with the other models.

[@TzaraDuchamp]

Yes, your code runs without errors with Gemma, but that model and Llama 3 8b can't handle the agentic aspect with the given code. They report 'Agent stopped due to iteration limit or time limit.'. This adversely affects the BTC price inquiry mail response. Mixtral 8x7b runs well and handles the agentic aspect. Llama 3 70b can become a bit congested (waiting list, though haven't had it with the API) due to popularity, so it's a good option to have. I would be interested in you exploring Llama 3 8b's agentic prowess.

[@samansaadzadeh1833]

Thanks for video, I was getting key error when test for the other email when it needs to use the def route_to_rewrite(state). look like the research_info key is not required for def route_to_rewrite(state).

[@user-wr4yl7tx3w]

Do you have an opinion on AutoGen?

[@jarad4621]

Can we use this process somehow to search the web and do research? Can't find this anywhere

[@ARSLANALI-fs8rw]

can you tell me what about taviely api how i can acces it?

[@FinGloss]

Can you show how to integrate with Gmail and to run locally with our own data ? Also, how to train on our own data. THANKS

[@josesalerno8586]

Hello Sam! Is there any way to do this task without using an API Key?

[@samwitteveenai]

You could run it on Ollama but the 70B needs a prety decent machine

[@niko_lev]

what's the flowchart tool you're using?

[@samwitteveenai]

Excalidraw

[@abbasbenterki]

Hello, I'm tearing out the few hairs I have left trying to adapt your excellent tutorial to create a workflow for detecting and tracking malicious emails (phishing). The process might look something like this: 1. Connector with an Outlook mailbox 2. Detection of received phishing emails 3. Collect received emails in Outlook into a file by batch or sync 4. Analysis of emails with Llama3/groq 5. Assign a score from 1 to 10 6. Classify phishing emails into 3 categories to create a security incident in EasyVista for those classified as critical 7. Tagging classified emails 8. Creation of a SharePoint folder that includes the metadata of analyzed emails 9. Reporting with BI 10. Tracing with LangSmith Kindly help. Thanks a lot.

[@stephenzzz]

Sam, question if you don't mind. My wife wants to have her sales information content incorporated behind a chat/RAG to answer questions from her content. Which system out there do you think would work best, that is low code for a non-dev. Ideally next part will be to access this via a membership website.

[@samwitteveenai]

I am not up really on all the latest no code solutions, and privacy would be a big issue here. I do think Notion has done some really nice cool things with their adoption of RAG across all your databases etc

[@dr.mikeybee]

When do you think someone will write a GUI design tool for LangGraph?

[@samwitteveenai]

FWIIW I have written something like this that handles CrewAI, AutoGen, and currently adding LangGraph. I will probably make a video at some point. There are still issues with regarding tools and complicated steps etc.

[@dr.mikeybee]

@@samwitteveenai That's great! BTW, I wonder why LangChain and LangGraph aren't Ollama-centric? Certainly most processing should be locally, and Llama3 is amazing. Do you think it's because they get funding from ClosedAI?

[@sanwellbeatz1630]

I like how you pronounce langchain

[@ps3301]

Instead of email, u should try to demonstrate langgraph using stock research agents!

[@samwitteveenai]

Have thought about doing this. Might take another look at it.

[@MsPolyaha]

Thanks so much for the explanation. I would like to add the logic for each step for more compréhensive humain understanding of this great exemple : step : ## Research Router change proposition in the prompt : "...Return the a JSON with a two keys 'router_decision' and research_router_logic to explain the logic behind it. use both the initial email and the email category to make your decision..." expected result : {'router_decision': 'draft_email', 'research_router_logic': 'The initial email is a thank you note from a customer, which only requires a simple response. The email category is customer_feedback, which also suggests a straightforward acknowledgement is sufficient.'}

[@nhtna4706]

What is groq’s role here ?

[@samwitteveenai]

It's serving the Llama3 70B model on their platform. Gives you much faster inference speeds

[@thedatascientist-lg4ls]

That's great, how about using an email from an account other than typing and passing the email prompt as it happens in the real world.

[@greendsnow]

Does anybody have unlimited groq api? Mine is not active.

[@PhattharaphonRomphet-kg3oj]

Thank you about the knowledge 🫶