Why Agent Frameworks Will Fail (and what to use instead)

Want to get started with freelancing? Let me help: www.datalumina.com/data-freel...
Need help with a project? Work with me: www.datalumina.com/consulting
You probably don't need an agent framework to solve your automation problem. In this video, I'll cover my approach.
👋🏻 About Me
Hi there! I'm Dave, an AI Engineer and the founder of Datalumina. On this channel, I share practical coding tutorials to help you become better at building intelligent systems. If you're interested in that, consider subscribing!

[@michaelirey]

While your critique of agent frameworks is spot on and compelling, it seems there's a misconception about their potential. Your custom system resembles langchain+langgraph, highlighting a need for deeper understanding before dismissing existing frameworks.

[@Fezz016]

Basically what this video is saying is this, "I do not understand the Agentic Framework Flow yet, so I will just critique it in the meantime because I do not understand it"

[@istvandarvas3372]

I mostly agree with everything. But there are two kinds of pipelines. The first one is when you have a finite amount of transformations, and the second one is when you don't know all the transformations in advance and need to delegate decision-making (in this case, you need an agentic approach). However, every pipeline can be represented as a finite transformation when you know it. for example classification. and t that is the key. So, if your pipeline is research-like, then you can't know it in advance; in other cases, you can.

[@matterhart]

I like this distinction. I build UI heavy pipelines with a lot of human input where there's a ton of variability/decision-making in both the E and the T, and only the L is known in advance.

[@daveebbelaar]

Good point. In those cases, agentic workflows make perfect sense.

[@ShpanMan]

@@daveebbelaar So you admit your title is extremely silly.

[@MrEnriqueag]

I agree with tour core message. But i dont think you've used langchain at the designed level or maybe dont know about langraph? Its not opinionated and you can (and should) orchestrate the flow however you want, you can make it linear, acyclical (every langraph example), you can decide the flow however you want, deterministally, defined by the LLM output etc. None of my agents are even driven by any default langchain agents, i have my own prompt, output format, tools etc. The framework is there to: 1. Standarize the way you interact with the models 2. Have a trackable verifiable analyzable way to build those graphs

[@EmilioGagliardi]

Interesting. Working on a CrewAI project atm and I found I was using a DAG approach to tasks because of my experience with Kedro. One task, one transformation, one output and keep working sequentially. In a nutshell, you're describing Kedro's approach and philosophy. Its just not fine-tuned for generative AI use cases yet. What I've found with multi agent apps is that I end up building tools that do all the heavy lifting and the agent Is used to generate a piece of data (like a query string) used in subsequent processing. The challenge is building guardrails to prevent an agent from going off the reservation when something doesn't work. If you give an agent access to a tool as simple as a search tool, if it gets stuck, it could end up calling the tool in a loop and there goes your credits. So we're still having to treat agents like toddlers... would be interesting to see your take on kedro.

[@drhxa]

We build agentic systems that design dags and those dags consist of LLM agents themselves. So for a complex task an LLM defines a DAG that is a set of LLMs which then sequentially solve the subtasks required. What are your thoughts on this concept of merging the sequential flow together with multi-agent system?

[@noduslabs]

Good take. All those frameworks are good for getting familiar with the principles but if you want to make a unique specialized product you need to code everything on your own. Probably you won’t need agents for some tasks even.

[@hailrider8188]

Cyclical/recursive algorithms are needed for many problems which in part, is what agentic frameworks attempt solve. Your sequential processing only paradigm is applicable only to certain problems.

[@ContextFound]

You have a solid point about agentic frameworks usually not being the right tool for tangible business applications. It's about automating the repetitive.

[@vidfan1967]

Agents are good for one-off activities, where you want the agent system to find a sequence of activities that gets the job done. Nice for non-coders or not-knowers. However, for a repetitive process, where you need to rely on the quality of output, you need to control every step and KNOW, that it will deliver a result you can handle in future steps. The issue with LLMs is the uncertainty they introduce, eg. unwanted bias, wrong facts, broken reasoning. Use the LLM only where it shines (understanding and generating text) - you would nit rely on the LLM to be good at math and use other functions instead. This same principle applies for a lot of steps, if you decompose the job into tasks. But you need to understand coding to properly do it (or use an AI to do it for you once and then write the task sequence for you with minimal LLM). And this is not even considering the high costs agent systems produce compared to restricting LLM use where it is beneficial - or understandability, how the result was achieved…

[@alexanderandreev2280]

langgraph + function calling + langsmith = production "LangGraph is a way to create these state machines by specifying them as graphs."(c) LangChain

[@Crates-Media]

Here's something you can help me understand, as an intermediate-level coder learning all of the nuances of AI/ML and their applcations. You're extolling the value of the directed acyclic graph approach towards data processing pipelines, to avoid sending data to earlier stages. As a fan of idempotency and functional programming, I _think_ that I somewhat understand where you're coming from in your premise. But in my studies of models, I'm also seeing a lot of buzz around the differentiation between methodologies of KANs vs MLPs. My question is this: wouldn't there be some value in using information uncovered later in the pipeline to refine what you're doing earlier on? For instance, let's say you're entertaining guests, and planning to serve appetizers. A very early step might be purchasing ingredients. Later on, you realize that not all of the guests show up. If we're just going to keep moving forward, we make more appetizers than are needed. The alternative: when less guests show up or RSVP, instead of making as many apps as your ingredients/plans dictate, you make less. Now you have less appetizers and you store or freeze the ingredients you didn't use. You _could_ make them, and freeze the unused portions. But by sending the information collected later back to an earlier step, you instead have the raw ingredients to use in other recipes instead. This is a really lousy and forced metaphor, but it's all I could come up with off the top of my head. It just seems like there's value in the concept. On a different level, isn't this just sort of a form of backpropagation? The ability to reinform earlier calculations with the results of later ones?

[@micbab-vg2mu]

Great video - thank you for sharing:)

[@brianhauk8136]

You said this is a work in progress, and I'm wondering if you've compared the results of traditional Mixture of Agents responses with your pipeline approach for various common use cases.

[@afazeli8361]

Awesome content, Keep up the good work

[@koen.mortier_fitchen]

Agreed. Take a simple airtable, input cell connected to an llm, and an output cell for its response. There you have your first step of an agent. The hours I lost on learning Langchain, Flowise, you name it

[@zanderrossouw1929]

I would LOVE to see your design pattern in depth

[@teprox7690]

Everything is great. I have built a few tools myself with Instructor. To really automate business processes, however, I see the problem with data protection. In the EU, I can't just put a complete e-mail into an LLM. How do you solve this? It would be great if you could shed more light on the subject of data protection! Thank you very much for your excellent content!

[@RealEstate3D]

Exactly what I build for my company. Small AI that discovers PII in a text that I want to give our offices for use in their intranet before sending it out to public LLMs. Beforehand I tried Presidio and octopii. Both use regexp. Got bad results.

[@stewartfarquhar1280]

great work. please publish the next tutorial. is there a github for the code?

[@user-bt6pp1dt4w]

Hey, looks very reasonable. Have you looked at prefect and its new controlFlow libraty?. It helps to manage this data pipeline pattern for LLM

[@teprox7690]

I think we will see special models trained for the agent workflows. Right now, they are trained with way too much knowledge for this workflow. Then the latency will also go down. I'm currently wondering why we haven't heard anything about this approach yet.

[@hannesg.3586]

Thanks for the video. What kind of whiteboard tool do you use?

[@anubhavtomar1384]

its figma

[@aimattant]

I followed through. Your system looks, great, with content creation processing with your AI generative pipeline. However, I think the point of Agentic, which is not there for sure, is to be able to work in a non-rigid system, the bottom-up approach, but where all systems are communicating together. So it is a build-up on pipelines that you have. Which is amazing by the way 😊 And putting them all together in a system that works autonomously. The idea is to get AI to the point where it will work as a team without instruction. The whole point is why Sam and all the others are building these huge systems now. One thing I want to get my head around is nongenerative AI; is content base. One thing I am seriously delving into now is API endpoints of all kinds with AI LLM model support. With some tasks, they are not required. But with many where data is involved, they are. Hope this makes sense. Not here to put a dent in your wonderful work. You are great at coding and putting the AI infrastructure together. Look forward to following along with you. AI agent workflows are the way forward now.

[@alvandal]

Great video, I am not big on KZbin, but this is the first time I see someone really understanding the current state of the tech.

[@crepusculoooo]

very interesting i was applying this kind of tasks in my company too like email reading attached files processing etc. is code avaible? thanks alot

[@JohnDoe-m6s]

@13:33 Please create an in-depth video on these concepts using the example that you are showing in this video !

[@Rusputin-eo6tv]

framework in the end is just how you organize your code.

[@yasminelmahdi2003]

Hey Dave, I'm working on my undergrad project research, and it's about AI agents, I have a couple of questions if there's a way could you share a way to contact

[@agnivamahata3870]

This works, but giving AI to take decisions are supercool, new libraries will emerge check agency-swarm. These works for data driven processes but first creative processes agentic systems make sense

[@dievas_]

Yeah, but you dont need frameworks for that, agents are just basic methods that call each other or tools. You also create some class for state management. Function calling is also easy. All those frameworks are opinionated and only adds bloat. Unless youre making something with 20 agents, but that would be nonsense...

[@toi500]

One of the worst videos I've seen this year. You ignored a very capable framework like LangGraph. Also, the fact you did not talk about hierarchical or sequential agent workflows and you ONLY focused in the multi-agent ones to present us "your case" is not serious. In any event, even a simple multi-agent graph can achieve a 100% success rate in resolving tickets. You don't need to hardcode all those steps at all.

[@jasonsuarez6439]

I thought the same starting the video but I ended up appreciating the effort to take a step back to first principles. It's refreshing and at least make us think about choosing a framework. If he was really dishonest, I could have talked about the biggest problem of those agentic frameworks... latency at scale

[@daveebbelaar]

I might be too ignorant here, but I haven't seen any of those frameworks implemented in production yet. I'm aware that I just scratched the surface here and that there are indeed many different agent workflows. My point is not to discard all of them but rather to mention that, based on my experience, for most business automation problems, building your own pipeline from scratch is typically better. What's your experience with agent frameworks, and do you have a good production-ready use case? I'd love to learn more about this because I really want to like them!

[@ShpanMan]

@@daveebbelaar Not a single person thinks Agents are ready for production. It's an early implementation prototype that gets better and better with each new AI model that comes out. In a few years it will crush any human and send you to retirement (in a good way).

[@bphilsochill]

Do you have a use case to share?

[@qredence]

Not 100% agree but largely clearly on our side! In fact the currents frameworks are not enterprise ready, unpopular opinion you seem to share too ! Imo Orchestration is the real key ;) but agentic systems with stateflow nested pattern connected to external tools/fabric/composio/else can be a part of solution The other key is RAGsystem, uncertainty first focus based framework that adds more complexity but as we think is that is almost mandatory for companies. Greate video ! You should follow closer Microsoft Fabric and Semantic Kernel

[@jonm691]

I really liked your video, and the message, but I just kept thinking that you'd just done LangGraph 'properly'. It has much more capability, more transparency, more enterprise considerations. etc. I'm not sure I agree that you must never under any circumstances loop back. All that means is that you leave open a need to fail a process and with additional context, get it resubmitted. Did I miss something?

[@andydataguy]

Step 1) understand how the tech works Step 2) use it to it's best ability Hasn't this always been the case with new technology for decades?

[@CookTheBruce]

Have you looked at VRSEN’s Agent-Swarm? Sounds like he avoids many of the pitfalls you describe here…

[@333SFx]

Langgraph

[@mikekidder]

state machines, state machines, state machines...

[@not_a_human_being]

existing frameworks are bloated, you have a point there, it's like they only make sense to people who created them. but I'm not sure what exactly is Celery adding here? extra complexity with no extra gain. Defining DAGs is fun, but usually doesn't add much value

[@NockyLucky]

This was so wrong

[@tonykipkemboi]

So what you're trying to say is it's simple to build agentic workflows, hence do it yourself and don't use the existing frameworks? If yes, your video/take would have been much better and persuasive if you dug into the implemetation of the said frameworks while pointing out the cons. Your example is very basic and doesn't even need an agentic process tbh. You can write a script to handle this like you did. All am asking is a more in-depth comparison and not a one-sided take.

[@LuisBorges0]

AutoGen and CrewAI I think are more experiments than anything else. I use Python as a Maestro... then some AI, when I need it's generative abilities, some playwright when I need web automation... but business run on logic not in a democratic way and not in a creative way. They incorporate creativity in some steps but that's just that. A procedure is the only way to go since ever or else we'll have unpredictability that goes against efficiency and bad for processes in business, in factories, and would make it impossible to have any type of Quality Control.

[@Bartskol]

This is exactly what i needed.

[@denisblack9897]

Another day - another wave of similar videos… I did not get this agents bullshit from the start(felt like data gathering scams to me), cuz i was chaining prompts when chatgpt came out. You gonna chain prompts manually if you are building something more than a useless demo.

[@mrprogamer896]

could you explain more about "chaining prompts manually"? Everything about agents seems confusing to me tbh.

[@scottc8274]

I think this is fair, for linear data flows and data transforms simplicity is best but for complex, multiphase transactional goals which contain conditional variables which could change during the flow you need a controller/manager which can identify and respond to these state changes. In addition ideally an orchestration agent wouldn't be bound to a set of pre agreed agents/services/API s but would be constantly optimising its flow - searching out better ways to reach its outcome. A solid Agentic framework should be able to switch between and explicit and discoverable transformation approach hopefully..

[@SSIAutomations]

I ageee

[@opensky1710]

My problem is having hard time to find girl friends.

[@mtrifiro]

Thank you for putting into words exactly what I had been feeling when trying these agent tools.

[@ShpanMan]

You are just looking for reasons why your career is not ending 🤣 Just accept it and be thankful AI will do all the work for us in a few years.

[@ali-om4uv]

Really cool idea!! You need to get rid of your erat8c teaching style tho. Especially with respect to quickly showing pictures jump g to Code etc.... Anyhow.... I would love to See the full fledged videos!!

[@daveebbelaar]

Thank you for the tip! Normally, my tutorials are structured and step-by-step. However, this one was not intended to be like that. But it's good to know! I will most likely publish a video on this approach in the near future.

[@SiyaNtombela]

Such a weird take

[@TheEtrepreneur]

critical thinking rocks!