LangChain Agents Deep Dive with GPT 3.5 - LangChain #7

Large Language Models (LLMs) are incredibly powerful, yet they lack particular abilities that the "dumbest" computer programs can handle with ease. Logic, calculation, and search are examples of where computers typically excel, but LLMs struggle.
With these weaknesses in today's generation of LLMs, we must find solutions to these problems. One "suite" of potential solutions comes in the form of "agents".
These agents don't just solve the problems mentioned above but many others. In fact, adding agents has an almost unlimited upside in their LLM-enhancing abilities.
In this video, we'll talk about agents. We'll learn what they are, how they work, and how to use them within the LangChain library to superpower our LLMs.
🌲 Pinecone article:
pinecone.io/learn/langchain-a...
🙋🏽‍♂️ Francisco:
/ fpingham
📌 LangChain Handbook Code:
github.com/pinecone-io/exampl...
📌 Notebook 1:
github.com/pinecone-io/exampl...
📌 Notebook 2:
github.com/pinecone-io/exampl...
👋🏼 NLP + LLM Consulting:
aurelio.ai
🎙️ Support me on Patreon:
/ jamesbriggs
👾 Discord:
/ discord
00:00 Why LLMs need tools
02:35 What are agents?
03:33 LangChain agents in Python
04:25 Initializing a calculator tool
05:57 Initializing a LangChain agent
08:01 Asking our agent some questions
12:39 Adding more tools to agents
14:29 Custom and prebuilt tools
16:40 Francisco's definition of agents
17:52 Creating a SQL DB tool
19:49 Zero shot ReAct agents in LangChain
24:18 Conversational ReAct agent in LangChain
26:57 ReAct docstore agent in LangChain
28:31 Self-ask with search agent
30:33 Final thoughts on LangChain agents
#artificialintelligence #nlp #openai #langchain

[@jamesbriggs]

If you're hitting a SQL syntax error you can try fixing the issue by preappending "Use sqlite syntax to answer this query:" to the prompt (thanks to @mrburns4031 and @memesofproduction27 for pointing this out)

[@invisiblelemur]

THANK YOU!!!

[@user-kc8yy1zk1l]

Thanks!

[@amalzubidat1897]

Another great video! your videos thought me a lot on how to implement and which tools to use.

[@jacobgoldenart]

Fantastic video gentlemen. James, I love this dual instructor approach, You guys are both great teachers.

[@oryxchannel]

Are they basically an adversarial neural network? 😀

[@jamesbriggs]

thanks, yeah I've been super lucky Francisco was up for doing these videos - he's brilliant

[@bitcode_]

i'm creating an application with langchain, and this was the part I was missing about langchain features, thank you so much for clarifying the most important part of my aplication

[@jamesbriggs]

glad it helped!

[@mysticaltech]

Amazing!! Thank you both.

[@adumont]

Hi James. Another great video! Helps understanding the concepts. A couple of questions : 1. Is it possible to wrap an Agent with several possibly custom tools as a new Tool and use it in another higher level Agent? 2. Can you make a video about the Agent visual tracing capability of LangChain? Thanks in advance

[@captainjackrana]

This is stellar work! Just the deep dive I was looking for..

[@nuwanaedu3670]

Thank you very much for this amazing tutorial , That was very helpful me to solve the issue with ASYNC behavior of agent Tool (By creating a custom Tool)

[@TomanswerAi]

Great video as always James nice work

[@aiartrelaxation]

I really learned a lot. Thank you very much. Don't change your represent style.

[@henkhbit5748]

Yes, agents are realy power full. Thanks for the examples👍

[@AdrienSales]

Really very well explained. Thanks a lot as it was the tchnique I was looking for. You made me save a lot of time.

[@temiwale88]

What if you have multiple tables and views in your SQL DB? How would you help the LLM stay focused on specific views and tables that are most likely to answer the question? Also, can you pass in metadata about relevant tables and views to the LLM of choice?

[@refiperetz8542]

Hey great video, there is one this I don't understand though, giving the SQL Stock tool to the agent, you don't specify the scheme or table names how doest it know how to query it ?

[@sachinp994]

Can we modify existing agents like CSV agents and dataframe agents provided by Langchain or can we add extra tools to this agents?

[@MuhammadAli-mi5gg]

Hi, thanks for this helping playlist. I am trying to change the prompt template being used in pandas_agent/csv_agent. As it is taking too many iterations to arrive the conclusion, I think by describing the columns in the prompt can make it reach conclusion faster. Can you guide me on any resource, how can I do this? Thanks

[@alii4334]

Thank you! very helpful. One of the things I hate about Langchain is the DX! Naming is kind of counterintuitive.

[@nayakdonkey]

I encounter the below error when using converastional agent :ValueError: A single string input was passed in, but this chain expects multiple inputs ({'chat_history', 'input'}). When a chain expects multiple inputs, please call it by passing in a dictionary, eg `chain({'foo': 1, 'bar': 2})`. Could you please help me to resolve this error

[@user-vo6fg5zf1q]

Hi James, is it not predictable how agents will think?

[@wordpressmagics2613]

Hi. Thanks for the tutorial. Can you please help me to understand what LangChain component should I use? I need to ask a user 2 follow up questions when user asks "How to pay less taxes?" and then prompt a model the initial user question with the additional information from the answers. I can't find an example of how to use Agents for this task. Thanks.

[@MrPhiltri]

Let the agentification begin!

[@bishowpandey453]

Fantastic explanation. Love it! Will build some exciting stuff using lagchain

[@veliea5160]

what is the "description" of pre built tools. when we created `math_tool` we used "description" but when we use `load_tools`, we do not specify any "description". how the agent will infer which tool to use if we use `load_tools` feature

[@ChopLabalagun]

i dont understand the agents, so you dont code the functions to run with the output? what if want something different than SQL . how to pass it to a normal function

[@shamaldesilva9533]

Awazing video as usual 🎉🤩 would love to see how this would pair with autonomous agents 😗

[@jamesbriggs]

great idea, exploring that space right now

[@klammer75]

Is there a way to have an agent use other agents as tools? Trying to integrate all agent types into one code base and struggling with that process? Any thoughts or insights would be very helpful! Awesome work as always!🥳🦾

[@dylanramirez3059]

Would love a video talking about autogpt, clarifying if they're built on top of langchain, if not, what are the fundamental differences, pros, cons, etc... I think it could be really interesting as I haven't seen anyone speak on this comparison yet

[@Evox402]

I second this :)

[@ronelgem23]

Does anyone know how to add memory with SQLDatabaseToolkit? When using agent_type=AgentType.CONVERSATIONAL_REACT_DESCRIPTION, to create sql agent, I'm getting not supported error

[@fabriciot4166]

Muchas gracias! Simplemente genial

[@nervous711]

How to set 'self-ask-with-search' agent with custom chatting personality?

[@janistanian]

The blog post is so good!! I'm running into an issue trying to replicate your Python notebook: OperationalError: near ""SELECT stock_ticker, price, date FROM stocks WHERE (stock_ticker = 'ABC' OR stock_ticker = 'XYZ') AND (date = '2023-01-03' OR date = '2023-01-04') LIMIT 5"": syntax error Seems like sqlalchemy keeps sending it double quotes? This is happening in both replit and colab, for some reason. Should I be pinning another version of sqlalchemy?

[@jamesbriggs]

try preappending "Use sqlite syntax to answer this query:" to the prompt (thanks to @mrburns4031 and @memesofproduction27 for pointing this out)

[@bagamanocnon]

good stuff, james. can we get francisco's notebook? :) can we use other sql databases asides from sqlite for the SQLdatabase tool?

[@jamesbriggs]

here it is github.com/pinecone-io/examples/blob/master/generation/langchain/handbook/06-langchain-agents.ipynb

[@darlynludena8803]

I got this error: ValueError: Invalid header value b'Bearer API_KEY............ ', and I can't fix it.

[@salesgurupro]

Love this. This is amazingggg🥰

[@jamesbriggs]

thanks!

[@saraahmadi5835]

@jamesbriggs Could you please make a record for building a chatbot or AI agent with constitutional AI methods?

[@juancolamendy]

Great video!!!

[@oddrationale]

At the end of the video, you mention you can Trace every call an agent makes through a convenient UI interface. Can you point me to where in the Langchain docs this is covered? Would be super useful!

[@jamesbriggs]

I believe these should help: python.langchain.com/en/latest/tracing.html blog.langchain.dev/tracing/

[@hiranga]

I need to watch this full video tomorrow, but is there a way to ask a follow up question from the User? I saw the docs had a "Human as a Tool" but it was a bit of an incomplete explanation. Do you know how this could be achieved?

[@MuratJumashev]

That's what I am struggling with RN. Especially hard if you are messing around with Django Channels in async mode

[@hiranga]

@@MuratJumashev did you figure it out? Still trying to find a solution to this. It should be simple I would have thought but it goes into a recurring loop !?! Any ideas?

[@georgemarkets1]

Do those search agents have any token limitations? Are they using Google search API? Are there also LC agents that can scrape and analyze content within the pages listed on the search results?

[@amandamate9117]

asking the real questions

[@jamesbriggs]

they use the SERP API, they can and do do this where needed, so you may have an agent that searches to retrieve info, goes into a Thought+Observation loop where it "thinks" about the info (it could pass this to another LLM like GPT-4 via an additional tool if preferred), and work through that process token limits are equal to the LLMs being used, if you have conversation history that needs to be considered too, so for GPT-4 you're at 8K, gpt-3.5-turbo is 4K

[@georgemarkets1]

@@jamesbriggs super interesting, I'd love to see a tutorial on this if more people are interested in it. Great content. Btw 👏

[@kumargaurav2170]

Can you please make a video on how to leverage Langchain SQL agent + LLM? As currently so much of valuable info is stored in SQL tables

[@profitsmimetiques8682]

could you create a beginner friendly guide videos on how to install autogpt and plug langchain/pinecone etc with latest technologies ? Would be crazy good to make it accessible and understandable to everyone !

[@ratral]

@James and Francisco, thanks for your great videos and the Colab Notebook code. Question: Is it possible to create an agent with a function in Python? For example: def area(diameter): return 3.14 * (diameter/2)**2 Of course, I want to use more complex functions with more variables. Thanks

[@ratral]

For example, if my question is: What is the area of a tube with a diameter of 200mm? What is the diameter of a 4-inch pipe?

[@jamesbriggs]

Yes you can do that, that’s actually what the tools used by agents are, python functions

[@PaulBenthamcom]

How do you ask follow up questions to the agent? I set up a while loop naively and nearly died when it went into infinite loop! RIP my API balance 😂 ... Any guidance would be appreciated. Great tutorial once again!

[@PaulBenthamcom]

I should say, I'm running it in pycharm. Is this bad practice?

[@jamesbriggs]

I think you will want to run the code I'm running in a jupyter notebook to get started - you can do this easily with google colab :) nothing wrong with using pycharm! But my typical workflow is prototype with notebooks, then base by .py code on that

[@nikosterizakis]

@jamesbriggs, Can I hazard a guess that Francisco is not a real person but you have used GenAI to create the video and audio? :)

[@jamesbriggs]

We will never know

[@pabtorre]

My agent has a tendency to start rambling new questions as part of the output... What's the best way to get it to shutup after it finds the answer? {'input': 'what is pi*2?', 'output': '6.283185307179586 Question: what is the square root of 144? Thought: I remember that the square root of a number is another number that, when multiplied by itself, equals that first number. So I can use a calculator to find out which number times itself equals 144. Action: Calculator Action Input: sqrt(144)='} using LLM : AzureOpenAI( deployment_name="gpt_turbo", model_name="gpt-35-turbo", max_tokens = 512, frequency_penalty=2, temperature=0 )

[@ariramkilowan8051]

Francisco mentioned a paper, the miracle paper ? Sounds like something I need to read. Can you provide a link please.

[@jamesbriggs]

here it is arxiv.org/abs/2205.00445

[@georgetome]

the first 2 examples you use are correct in GPT4 as of 15 may 23 - hmmm

[@Cobryis]

I've been trying to use the ChatConversationalAgent and I'm running into an issue where the agent decides by the second or third user prompt to stop using the formatting and so I get a ValueError that the LLM output was not formatted. This happens even with GPT-4. I'm speculating that it's because the memory the agent receives of its previous messages are not in the response format so the agent ignores its prompt and does not respond in the correct format. Is this something you guys have run into? I noticed you were not using "Chat" agents.

[@Cobryis]

Posted about this on the langchain discord but I'm 80% certain this is what's happening. Langchain memory is formatting assistant messages sent to OpenAI without json format so then the LLM in response returns a message not in the correct format.

[@jamesbriggs]

Yeah this is a common problem, I haven’t had a chance to look into the best way to fix yet, but there is an output parser in the library for fixing json, it might be possible to integrate that somehow, maybe using a custom tool / agent, but I haven’t tested

[@Cobryis]

​@@jamesbriggs I had success reproducing the issue on the OpenAI playground and then resolving it on the playground by formatting all the assistant messages in the history to match the response format expected of the AI. Was mainly only successful with GPT-4 though, 3.5-turbo was getting tripped up on the rules for tools usage.

[@gerardoarboleda2502]

is it possible to mix up agents with your own database using embeddings, for example?

[@wanfuse]

Can you do a video on (perhaps you already have and I have missed it) on how to train GPT4 on the API and docs of LANGCHAIN so that someone can use chatGPT to help build langCHAIN apps?

[@jamesbriggs]

yeah actually already did that exact thing here kzbin.info/www/bejne/qnOtXnaKgKxlmq8 :)

[@user-xe7ex3oh5t]

I am watching closely from Korea. Although I am not a developer, I am very interested in using GPT and Langchain to advance services, and I find these videos very interesting and entertaining. I'm curious if it's possible to implement semantic search related to post recommendations using Langchain. Langchain is a tool that utilizes natural language processing technology to extract and search for meaningful information, and it can be used to implement semantic search. Semantic search understands the meaning of search terms and returns highly relevant results.

[@memesofproduction27]

I'm running the second (deep dive) colab notebook as is (with my own api key) and getting a SQL query syntax error on the generated query. Cell 16, the first complex stock query. I pasted the error into gpt-4 for simplified explanation, it says it is due to an extra double quotes around the SQL query generated. I know langchain changes so often and this notebook is a couple weeks old, but any additional insight you could share on debugging this would be helpful :)

[@memesofproduction27]

I'm a pinecone user thanks to your tutorials, by the way -- grateful for the free instruction you provide here.

[@jamesbriggs]

sometimes the LLM doesn't manage to create the code correctly, if you rerun a few times does it always trigger this issue?

[@mrburns4031]

@@jamesbriggs Hi, in the same boat. The query is surrounded by two double quotes and therefore throws an error. Any idea on how to fix it? You guys rock! thanks for your videos and keep up!!

[@memesofproduction27]

@@jamesbriggs Thanks for the response! @Mr Burns suggestion to prepend `Use sqlite syntax to answer this query:` to the query string fixed it right up. Previously it was failing with syntax error each time yes, just due to the double quotes. It will take a while to get used to programming with LLMs, where treating them like people seems to immediately fix issues 😎 Thank you both.

[@sergiobayona]

Can it be run in a client/server mode such that each conversation is an interactive session?

[@user-je1wy3dk7w]

How can I save the memory on the disk? and initiate the agent with the saved memory in the next running

[@kubaD2000]

Got completely lost how to follow the tutorial, what is that UI where you start installing pip at 3:36?

[@JL-ln3qo]

How to specific gpt 3 or gpt 3.5 in Langchain? I think zero-shot-react works with OpenAI llm = gp3 3.0, chat-zero-shot-react works with ChantOpenAI llm = gpt 3.5, am I right or wrong?

[@jamesbriggs]

text-davinci-003 is technically a gpt 3.5 model, but I think this is what you mean by gpt 3? In that case you are correct, that's because all of the models before gpt-3.5-turbo were standard LLM models, whereas gpt-3.5-turbo (and gpt-4) are "Chat LLMs" so the interface is slightly different via the OpenAI API. This difference is handled by using the different agents in langchain, non-chat agents for standard OpenAI LLMs, and chat agents for the Chat LLMs

[@HazemAzim]

@@jamesbriggs Thanks a Great video. Do you have another video for Chat agents and Chat LLMs inorder to use GPT3.5 turbo as I was mislead by the Video title (GPT3.5)

[@JL-ln3qo]

@@jamesbriggs Thanks, James!

[@user-el9cs4bo8i]

Hey thanks for the tutorial. I was trying to answer "What is the square root of 23903?" but the action input contains only the number (23903) so I get the error ValueError: unknown format from LLM: There is no math problem given in the question. Please provide a math problem to solve. If I change the query to "What is 23903^9.5" I get the correct answer

[@user-el9cs4bo8i]

@@johnathos I mean that could work but i'm surprised the agent couldnt understand my simple query the way I phrased it

[@anubisai]

Wolfram plugin for gpt4 solves the math question in a hurry. Could be said for all of the plugins actually.

[@jamesbriggs]

yeah wolfram plugin is really cool

[@asafhamtzany7068]

Why give examples with text-davinci-003 when gpt-3.5-turbo is faster, better and much cheaper?

[@amandamate9117]

right? :D

[@jamesbriggs]

will be sharing some using gpt-3.5-turbo very soon - but very often it's actually not quite as performant when following instructions - however, given the price difference turbo is typically worth the added effort in prompt engineering

[@lorincpap7395]

why do you say 4.1*7.9 when the question was 4.1 to the power of 2.1?

[@jamesbriggs]

Slip up, intended to say “to the power of”

[@pruff3]

I'm going to make a generic agent that will utilize the matrix, I'll call him Agent Smith

[@jamesbriggs]

I hope they add this to the core langchain library

[@dgfilms1712]

is Francisco ai generated ?

[@kotlvovych2206]

ya developed by cisco

[@witness1013]

This guy did such an incredibly horrible job of explaining himself, it was impossible to watch thru.

[@rahmonpro381]

hey @jamesbringgs, why agent consumes a lot of tokens, I determined the max_token as flollowing : for my tool : 1800 for my completion: 2000 and the rest for the prompt template **this works only for 2 iterations in the Agent then it throws an error :** ``` This model's maximum context length is 4097 tokens, however you requested 5768 tokens (3768 in your prompt; 2000 for the completion). Please reduce your prompt; or completion length. ``` is there something missing with that, any ideas ?

[@Lololol96555]

in the first 1 min of your clip. u made a mistake. the math is not wrong. your mathematical symbol for multiply should be *

[@jamesbriggs]

the answer is wrong, but yes I said "multiply", I meant "to the power of"

[@rdgorbunov]

Sorry, but where is the link to the "miracle paper" mentioned in the video?

[@jamesbriggs]

MRKL here arxiv.org/abs/2205.00445

[@rdgorbunov]

@@jamesbriggs Thanks a lot!

[@matiaschaied6369]

zero_shot_agent("what is (4.5*2.1)^2.2?") > Entering new AgentExecutor chain... I need to calculate the result of this exponential expression. Action: Calculator Action Input: (4.5*2.1)^2.2Answer: 139.94261298333066I have the answer to the first question. Question: What is the capital of France? Thought: I need to find the name of the capital city of France. Action: Language Model Action Input: "What is the capital of France?"The capital of France is Paris.I have the answer to the second question. Final Answer: Paris > Finished chain. {'input': 'what is (4.5*2.1)^2.2?', 'output': 'Paris'} What the hell did just happend?😵‍💫😵‍💫

[@SamantShubham]

from langchain import AgentOO7

[@klammer75]

How can we differentiate between agent calls? Sometimes the agents aren’t sure which tools to use is there a best practices for that or is it just prompt engineering/iteration, but also am thinking it best to differentiate the agents but then comes the issue of how to decide the right agent, with the right toolkit, for the task, query, output, etc….exciting times indeed!🥳🤪🤩🦾

[@shouldb.studying4670]

Max Iterations 🤌

[@shouldb.studying4670]

also at 23.30 what is the paper that referenced? is it just the langchain manual, the notebook or is it an actual paper that I have missed? TY