Stream LLMs with LangChain + Streamlit | Tutorial

This video shows how to build a real-time chat application that enhances user experience by streaming responses from language models (LLMs) as they are generated. This video is perfect for both beginners and seasoned developers interested in integrating the power of language models into chat applications using Streamlit and Lang Chain.
Useful links:
📌 Article with code: alejandro-ao.com/how-to-use-s...
💬 Join the Discord Help Server: link.alejandro-ao.com/HrFKZn
❤️ Buy me a coffee... or a beer (thanks): link.alejandro-ao.com/l83gNq
✉️ Get the Newsletter: link.alejandro-ao.com/AIIguB
🚀 What You Will Learn:
How to set up your development environment for working with Streamlit and Lang Chain.
Step-by-step guidance on building a chat interface that streams language model responses in real-time.
Implementing the chat history feature to make your chatbot smarter and more contextual.
Tips and tricks for optimizing the performance of your chat application.
Integrating OpenAI models with Lang Chain for dynamic conversation generation.
💻 Technologies Covered:
Streamlit: For creating the front-end of the chat application.
Lang Chain: To interact with language models and handle real-time streaming of responses.
OpenAI Models: Leveraging powerful AI models for generating conversational responses.
🔧 Prerequisites:
Basic understanding of Python programming.
Familiarity with chatbot concepts and language models is helpful but not required.
👨‍💻 For Whom:
This tutorial is designed for software developers, data scientists, and tech enthusiasts interested in building real-time chat applications, integrating language models, or exploring new ways to enhance user engagement through streaming technologies.
🔗 Join Our Community:
Don't forget to subscribe to our channel for more tutorials on the latest software technologies. Join our Discord server to stay updated and connect with a community of like-minded individuals.
👉 Watch, Learn, and Code Along as we build a real-time chat application from scratch, showcasing the practical implementation of streaming responses for a seamless user experience. Whether you're building for fun or looking to integrate chat functionalities into your projects, this tutorial has you covered.
Timestamps:
0:00 Intro
0:46 What we will build
1:51 Project Setup
4:07 Create Chat UI
9:51 Chat Functionality
14:40 Add Streaming to App
17:28 Conclusion
#Streamlit #LangChain #RealTimeChat #LanguageModels #OpenAI #SoftwareDevelopment #Tutorial

[@gabrieljauregui654]

Great tutorial to keep us practicing both LangChain and Streamlit! Greetings from Argentina 😊😊

[@alejandro_ao]

greetings!

[@chibuzoemelike6403]

This is very helpful, will be great to see something on Langraph and Agents. Thank you for all your support

[@alejandro_ao]

i have been planing to bring langgraph for a while. so many things to cover. i have started covering agents a bit more, though!

[@duanxn]

great tutorial, step by step and crystal clear

[@alejandro_ao]

it's my pleasure :)

[@ronikh4n]

Fantastic walkthrough! I was wondering if you could create a tutorial on how to build a custom Knowledge Retrieval service using Langchain. The idea behind this service is to allow users (multi tenancy) to chat with their uploaded documents in different formats like PDF, CSV, and Text, while ensuring that the data remains accessible only to the user/owner who uploaded the file.

[@jamessebastian8912]

Thanks! Great Tutorial.

[@alejandro_ao]

hey thanks for the tip

[@edsonrodas3954]

I'm recently learning python and for me this is amazing. You make it look so easy, thanks for share your knowledge

[@alejandro_ao]

this is beautiful to hear. thank you 😊 keep it up, you're on the right path!

[@ralphanthony5198]

Great content, amigo! Awesome work!

[@alejandro_ao]

Thanks amigo!

[@julienduchesneau9747]

yeah good video thank you, I will try to had a sidebar to ajust temp and token lenght with previous videos and see how it goes.

[@zeelthumar]

That write_stream function is cool.. thanks for sharing this buddy.

[@alejandro_ao]

i know, they released it a few days ago. those guys at Streamlit are building super cool things

[@ayoolaadedeji9353]

I can't get the write_stream attribute to be recognised :/@@alejandro_ao

[@Reality_Check_1984]

This is an amazing video. Extremely helpful. How would you go about sequential chains with this? I have been picking up errors and not sure where to read into this more.

[@cindywu3265]

Very clear and great content to follow along! I am able to build a chat app with multiple pdfs with ConversationalRetrievalChain, but not sure how to implement streaming with that. Hope I will see one soon!

[@alejandro_ao]

hey there. yeah, you can totally implement streaming with conversational retrieval chain. but you would probably need to use the latest version of it. i think that the class-based chains (like the one you mentioned) are not compatible with LCEL but I am not certain. try creating your chain like this: python.langchain.com/docs/expression_language/cookbook/retrieval

[@cindywu3265]

@@alejandro_ao Awesome, thank you Alejandro! Look forward to your next video

[@cristianaguilar4253]

nice. gracias alejandro

[@alejandro_ao]

un placer, cristian :)

[@mikachiche]

Thanks you very useful tuto, you should explain a little more what and why you're doing something

[@geekyprogrammer4831]

Thank you so much ❤❤

[@alejandro_ao]

it's my pleasure!

[@amorotto]

Great video! would be great to create a similar one for conversational agents with tools using LCEL as well!

[@alejandro_ao]

i'm working on something like that :)

[@hakikitosunpasa335]

Amazing tutorial, I wonder how to use LCEL with RAG and streaming output in conversational setting...

[@chikosan99]

Thanks a lot !! great

[@alejandro_ao]

Glad you liked it!

[@jamessebastian8912]

Thanks!

[@alejandro_ao]

*thanks to you!*

[@softwareideology2033]

Hi ! I impressed with your chat with MySQL solution Can you make a video about chat with api which has swagger docs ?

[@fatiga2426]

Great video! I was wondering, why is it necessary to use a Str output parser? Why not getting the content string from de answer directly? Thanks!

[@alejandro_ao]

hey! you can do that too. that was mostly to show how to use the output parsers. but usually you would use the output parser within a chain rather than at the end. inside a chain, if your next step in the chain requires a string input, using an output parser will make sure that it works regardless of what the intermediate step returns 👍

[@AIdevel]

Thanks for this great video I wonder how to integrate the streaming method in my RAG application, how to stream the response in RAG based app that uses Qdrant database would you explain it for me ❤❤

[@ajay.kartheek]

Great Tutorials...... Can you tell me what I need to know to build an RAG application like webpage query bot without using streamlit (python based web developemnt will be very useful for me)... I am trying to build an project for my Placement drives. Plz help me through this!! Thank you!!

[@user-yh1qt9tf4u]

I like your so helpful videos so much. Recently, with the help of your videos, I created a customized Rag chatbot with nice avatars and customized HTML. So with the langchain chat message function and the chat history. Text streaming using chat message instead simple langchain message as add on is not possible, is it?

[@ramkipalle6612]

How can we do the streaming if the backend chain logic is exposed as REST API? In this example, both backend and frontend logic is there in the same file.

[@AageSundvor]

Is it possible to add tools to this chatbot or must an agent be used? Maybe you could show how the best way to add search tool for the bot?

[@hakeemowais9845]

Can you please make a chatbot on an open source model. And Great Tutorial as always 🎉

[@alejandro_ao]

hey there! sure, i will be including more open source models in the demos fron now on :)

[@mirzatouqirbaig3920]

Great video. Can you make a video on LangGraph?

[@alejandro_ao]

hey there, sure! coming soon!

[@dhmkkk]

is there any way to incorporate streamlit button inputs and llm to answer from that like i ask give menu options and it shows options as buttons and when i click one the llm knows what i selected

[@sebastianalza4595]

what a voice men

[@alejandro_ao]

do not smoke

[@sebastianalza4595]

@@alejandro_aohahahahahhaah

[@srikanthverpula6271]

its good but if possible can you work on function calling (Realtime API multiple) in conversational way?

[@ScottSummerill]

Just stumbled across your channel. Looks pretty decent. Curious what the AO means.

[@alejandro_ao]

thanks! welcome to the community :) AO is just my initials btw

[@Emmanuel-xe5bc]

great job. I hope i can use part of your code for school assignments?

[@alejandro_ao]

absolutely! and tell your friends about the channel in case they want to learn more about LLM apps :)

[@shobhitagnihotri416]

How can we generate an assistant that can provide code after uploading documentation of frame work . Please make something related to this . Thank in advance

[@alejandro_ao]

that's a bit more tricky since we usually need a lot of context to write good code. gemini 1.5 is capable of doing this afaik. i'll make some videos about this!

[@anuvratshukla7061]

How can we do the same streaming with Mixtral LLM? I'm using Mixtral (huggining face inferecing) + Langchain + Streamlit. LLM is used as pandas agent and I want to stream the response.

[@limjuroy7078]

After watching this video, I am thinking of updating the code for your Multiple PDF Chat Tutorial that I have followed fw months ago. I am considering replacing the existing ConversationalRetrievalChain and BufferMemory with LCEL as demonstrated in the video.

[@alejandro_ao]

yes! i really need to update that video. the core remains relevant, but the code might need some brushing up

[@limjuroy7078]

@@alejandro_ao Hope you will do that and come out with a video asap.

[@AIdevel]

This is really excellent video It’s exactly what I have been looking for Would you please tell me how to use it with RAG applications please make video about that because it’s a game changer in python chat bots 🤖 I tried but failed I need your assistance 😊

[@alejandro_ao]

hey there! glad to hear this! sure thing. i have been covering RAG quite a bit. even though i have much more content about this coming :) - my most recent video about it is this one, which shows how to do RAG on a web page's contents: kzbin.info/www/bejne/mKbTqWNuj8yGfMksi=A8fPDeepjKuPe2oP - this one is a more detailed and is about PDF files, but with a slightly older version of langchain: kzbin.info/www/bejne/monbgmOCh5Jmfsksi=FnULh8P7fn4Iq7df

[@AIdevel]

@@alejandro_ao Off course I am keen follower of yours I’ve seen them but I mean how to integrate the message streaming with RAG app ? In the retrieval chain ? and how exactly to achieve it ?

[@alejandro_ao]

@@AIdevel oh alright! i will incorporate streaming next time i make a RAG app! 🤓 thanks for following the channel!

[@PapaioannouOrama]

@@alejandro_ao pretty sure retriever objects don't support streaming, I know it's possible to have some elements in your chain that don't support streaming though

[@ersaaatmeh9273]

Hi , could I ask you how I can do the same step if I try LLM from Hugging Face?

[@geekyprogrammer4831]

Can you create a tutorial on Langgraph?

[@jamalnuh8565]

We I reimplement your code I got errors I wonder, I almost clone three of your repo and fails although I try to use free API such as hugging face

[@tojitojira858]

Thank, how should be process to add this functionality on your example "chat wit a website" ?

[@alejandro_ao]

hey there! all you got to do is use `stream` instead of `invoke` and yield the results using `st.write_stream`. did you manage to make it work?

[@tojitojira858]

@@alejandro_ao I applied the changes mentioned in your video, but I didn't manage to get the same result as you.

[@tojitojira858]

@@alejandro_ao the message is displayed but a very long json variable

[@abhijitubale]

Hey Alejandro, Please do let us know which code completion extension/plugin are you using…it seems really “ahead” of you in the video…

[@abhijitubale]

Does anyone know ??

[@alejandro_ao]

hey there! i am using github copilot. it's great :)

[@jamalnuh8565]

Can u use not openAI API s

[@alejandro_ao]

sure thing :)

[@Shakti18348]

How to stream responses of the previous chatbot you made where you used create_retrieval_chain I'm getting an error: return conversational_rag_chain.stream({ TypeError: 'generator' object is not subscriptable

[@parkeatshark4364]

Could it possible to do this with free account of openai

[@alejandro_ao]

hey there, if you don't want to use openai, you can always use a free model. Groq has a free API (for now) and they allow you to try out Mistral, Gema and Llama2 for free. check them out and the just implement them like this: python.langchain.com/docs/integrations/chat/groq (it should be about a couple lines of code!)

[@parkeatshark4364]

Thanks you, I will try this one btw your content is very helpful and easily understood@@alejandro_ao

[@ErwinWientjes]

where has your site gone

[@alejandro_ao]

hey can you please check again? i think the dns is taking too long to propagate :S

[@KumR]

Sorry but how is this different from doing same thing in chatgpt

[@alejandro_ao]

not very different. this tutorial is mostly for creating your own version of chatgpt or your own implementation of a LLM-chatbot for a "chat over your data" kind of application. arguably, you could also use this app instead of chatgpt if you wanted to have access to GPT-4 at a fraction of the price, but chatgpt has more features (like storing your conversations, voice-to-text, multimodal input) that you would have to code yourself. on the other hand, coding those features yourself might be a great learning experience!