The 4 Stacks of LLM Apps & Agents

[@donatocapitella]

I love when people manage to describe an entire landscape or stack is such a direct and easy to navigate way, placing all the pieces of the puzzles together! Thanks for your videos!

[@stonedizzleful]

Great video. Was aware of all these different components of LLM apps but the way you've broken it down is excellent!

[@FeliciaXiao]

I cannot express how much gratitude I had for this video. Super clear and informative.

[@SpenceDuke]

I am thankful for your channel. You're helping me prepare to excel in my current role. I like this framework, its similar to how I sort of thought about things but not as explicit.

[@Diego_UG]

It is understood very well, and as you explain it, they look like modules in code, but I don't know if in practice this can be done in modules, I see in your videos all these separate features, such as the context compressor and filters, but could you show us in a demo of how to make an agent that has all these components well organized? It would be great to see how you extend the medical agent that you showed us in a previous video, which already has some of these parts, so that it has all 4 stacks, such as adding the context compressor and filters, in addition to the memory with a base of vector data like qdrant, they are just ideas. Thanks for teaching and sharing, greetings.

[@samwitteveenai]

This is more a framework for thinking about the various parts and how you should approach putting something together. I have been working almost exclusively on Agents now since April and LLM apps for a few years. The putting things together is certainly changing as new frameworks like Autogen and others come out. A lot of the processes I tend to run as micro services as well. but the thinking about them is pretty stable. I am planning a whole series of videos on agents and to go through the mechanics more than just the "ohh look another new agent" videos that I see a lot out there.

[@bingolio]

@@samwitteveenai perhaps a minimal implementation showing all the separate stacks?

[@mytj228]

Wow what a discovery I made by clicking on your video! Subscribed and liked. What a great content you have.

[@jayasiriasa9080]

Dear Sam, thanks a lot for sharing very informative video with a framework by putting all relevant items together. Waiting for a series on effective usage of agents. Thanks.

[@kavachchandra3123]

This is simply amazing! Would love to see all of these models explored further in their individual videos where in short examples and applications of each model is being explored. Snippets of code would help but I see that its more of a thought process and way of thinking so analogous explanations would work as well. Looking forward to something similar.

[@HaseebHeaven]

Here is a summary of the video in bullet list form with emojis: - 🧠 Intro: Overview of how LLMs can be used to build different types of applications and agents - 🧱 LLM Stack: LLMs organized into different "stacks" or layers that each handle different tasks - 🕵‍♂ Search/Memory/Data Stack: Handles data retrieval, question answering, search 🔍 - 🧠 Reasoning and Action Stack: Handles tasks, goal-oriented dialogues, decision making, automation 🤖 - 👨‍👩‍👧 Personalization Stack: Handles personalization, user preferences, conversations over time 🗣 - 🤝 Overall: The video discusses how LLMs can be organized and developed to handle various tasks through different "stacks" or layers of functionality.

[@LaHoraMaker]

Superb video! Regarding the reasoning stack, I would love a future video Autogen agents, testing different Open source models as coordinator of models with different capabilities like Mistral, Fuyu, SQLCoder or Pandalyst

[@qchaldemer]

This is a fantastic framework, thanks for sharing it. You are mentioning towards the end using LLama to get data out of a database, do you have any videos for that? I have seen the ones for using LLama 2 with RAG using vector store but not necessarily querying a database a bit similar to what OpenAI codecomposer would do

[@tekrunner987]

He wasn't talking about Llama but LlamaIndex, which is a library of connectors between LLMs and various sources of data.

[@collinshill4821]

Can you explain how you're using the word 'heuristics' with maybe an example?

[@atchutram9894]

Thank you. What kind of problems can we solve with this? how to think about ideas for such a project?

[@ai_product_manager]

What are you thoughts on using language models for data engineering. At first I thought that LLMs would be overkill for data engineering, but then I realized that I should be thinking about a small language model since cleaning data is a relatively small circle of tasks that an LLM can do. Combining that will search, I have a hunch that entire data pipelines can be replaced with a series of small language models. Thoughts?

[@bingolio]

THANK YOU. PLS share your related Google Talks etc.

Thanks Sam for this insightful video, once again. The reasoning part is definitely the part requiring more work and it can bring significant differences in an app. Working on Kyroagent I can translate your video to a very similar structure. Also, one important layer that I think is missing here, is the Interface. How the user will interact with the app and agents is a key element in building apps. not only UI but the flow and refining how we can dynamically build interaction with a AI based app. A lot to come in this area too.

[@tekrunner987]

This framework is probably not perfect, but it seems like a really good starting point to think about LLM apps systematically (which I find isn't easy to do at all when new tools, libs, methods, models and so on are coming out all the time). If I may offer a couple suggestions : * I feel like "LLM stack" is a bit of a misleading name, since LLMs may be involved in all stacks. I think it'd be better to use a name that describes the function of that stack (like the other 3), though I can't think of a suggestion just now. * For production-ready apps, we'll probably need some "meta-stacks" as well. I can think of at least security (I don't even know what that would look like, but LLMs definitely present some very specific security challenges) and observability (ensure that each element of the stack performs as required; seems especially important when you start using multi-step pipelines for RAG, agent interactions, etc.)

[@joaooliveira7051]

Very clear an usefull! Thanks

[@zd676]

I would also point out there should be a Responsible AI stack, AI / Data security, compliance, governance, etc.

[@theptrk]

"What reasoning is the agent going to use"

[@andrestorres7343]

with reasoning he is referirng to the different techniques of promoting there are, .e.g., chain of thought

[@tecnopadre]

Great video. Thnks

[@SonGoku-pc7jl]

i like so much this video! manual writing all sentences in fisic paper jaja xD good organitzacions, yes :)

[@kwtan3814]

This is gold! Thank you Sam

[@samwitteveenai]

Glad this is helpful, I wasn't sure if this kind of video would be useful or not.

[@sivi3883]

@@samwitteveenai This is absolutely very helpful! Please provide the videos on your google talks as well please!

[@ricardocosta9336]

lets goooooo! good job mate

[@micbab-vg2mu]

Great video - thank you.

[@bakrianoo]

Waiting for a video about how you can manage your time to produce these gems !! Special type of drugs 😂

[@samwitteveenai]

lol finding the time and a quiet place to record is my biggest challenge when I am traveling (like I am currently). This one is actually taken from part of a live talk I gave early in the year so was easy to reuse it.

[@bakrianoo]

@@samwitteveenai I wish you all the best in your life. But preparing for the topic and making the slides are critical parts here. How can you always be ready to do this besides your main work?

[@ackiamm]

Thanks Sir

[@carterjames199]

Please convert the google talks

[@alx8439]

RP - role playing - should be considered as a kind of mastrubation and publicity banned :)

[@klammer75]

You’re the best Sam! Thank you for this and keep bringing AI gold to us digital peasants🤩😎🦾