Stanford Webinar - Large Language Models Get the Hype, but Compound Systems Are the Future of AI

[@labsanta]

00:14 - The future of AI lies in compound systems, despite the hype around large language models. 02:27 - AI's future relies on integrated systems, not just standalone models. 06:50 - Focus on entire systems rather than just individual model components. 09:08 - Exploring diverse methods for model output generation beyond basic token selection. 13:22 - Sampling and prompting are crucial for AI system behavior. 15:32 - GPT-3 showcases advanced in-context learning for various tasks. 19:36 - Model performance varies significantly with prompt framing. 21:23 - Understanding AI requires a systems thinking approach that integrates models and prompts. 25:14 - Optimizing language model prompts enhances flexibility and performance. 27:07 - Systematic thinking enhances language model performance via optimization strategies. 31:03 - Cost constraints necessitate efficient system design for AI models. 32:57 - Future AI will involve complex systems rather than just large models. 37:01 - Future AI advancements hinge on diverse scaling methods beyond unsupervised training. 39:00 - Future AI will focus on compound systems over standalone language models. 42:37 - Complexity of AI systems will increase, drawing parallels to evolving technologies like Google Search. 44:33 - Language models will evolve, impacting society in both positive and negative ways. 48:17 - AI systems require careful oversight to prevent unintended consequences. 50:10 - Navigating AI development requires clear goals and understanding risks involved. 53:59 - Starting with proper software systems avoids pitfalls of prompt templates. 55:49 - Focus on systems, not just models, for effective AI development.

[@dr.teerakiatkerdcharoen2338]

Thanks so much. 🥰🥰🥰

[@christian.adriano]

Saved me 58 minutes. Thanks!

[@rakeshd7131]

What model did you use to summarize the video? 😄

[@flybyray]

⁠@@rakeshd7131the answer maybe „myself - I am the model“ 😂

[@NicholasWilliams-uk9xu]

Backpropagation and hardening existing knowledge, rather than generating new knowledge is not innovation. This approach doesn't solve new problems and merely combines existing ideas. True progress requires real-time autonomous "burn in" of weights when neuron output acceleration aligns with reward detection acceleration. Core rewards are simple goals (e.g., temperature), while intermediate rewards are patterns that accelerate with core rewards detection. These patterns are multiplicatively burned into memory and influence neuron weights to achieve goal convergence (when neurons output accelerates with these pattern detection accelerations = weight update factor). The system learns autonomously without pre-labeled data, using a multiplicative burn process to adjust weights based on reward measures acceleration convergence, in real time. Pattern detections are intermediate reward measures burned into memory through temporal acceleration convergence with core reward measures. Nodes in the network are tuned by this convergence as well, aligning core rewards, intermediate rewards, and network behavior. The appetite function increases activity based on inverse acquired resources, guiding the system's optimization and stopping it when acquired resources are high, to shift focus on other core reward accelerations and neuron acceleration convergence. The system forgets pathological behavior through negative acceleration, mutating patterns into new ones. Reward measures are culled based on their temporal acceleration convergence, leading to automated segmentation of reward systems. The network uses patterns as reward detection, shaping neuron weights when they accelerate with patterns. High-level intelligence involves dynamically generating intermediate reward drivers (patterns from other patterns, by multiplying patterns to find commonalities to drive neurons towards variants that have this commonality, subtracting them to find differentials to drive neurons weights changes down other optimization gradients) Once a neural network has sufficient inference capabilities, it can mutate intermediate reward mechanisms, allowing for dynamic procedural generation of reward drivers that further tune the network. To optimize neuron activity towards maximizing possible pathways towards reward detection acceleration, which then these inferred intermediate rewards are multiplicative burned further in or deleted as they converge with core reward detection acceleration. The system distinguishes between fitting existing knowledge and inducting new knowledge, focusing on goal maximization through convergent amplitudes and acceleration.

[@marcinwk]

Trying to get serious about LLMs and how to use them appropriately and this lecture just jolted me with zap of electricity! What a wonderful, thoughtful content and what clear and articulate delivery! I just might replay this lecure to learn how to give a talk about a topic, any topic... Thank you sir!

[@HieuTrung-y7z]

This is amazing lecture! Concepts are extremely well articulated. Thank you Professor Potts for hosting this webinar. Looking forward to more of such high-quality content.

[@superfreiheit1]

Should create a coursera or edx course

[@Andre-mi6fk]

What a great lecture this was!! Super important for anyone building AI Products with LLMs. Even if you think you know the material, it is good reinforce the best practices.

[@juanjoserojasconstain6561]

This was great! I finally get what was the interesting thing about in-context learning and emergent capabilities. Despite of being trained just to predict the next token, the model can learn perform NLP tasks (summarization, QA), without further training. Just from the right prompt. Before that, any model should be trained specifically for one of those tasks. 14:25 Reflecting on his idea of systemic thinking (7:23) is a must if you want to build applications with LLM, as he shows in 29:18. Using the same model (GPT-3.5), we can get a 20% performance boost just from the right prompt-optimization system 31:23. The questions were also very though provoking 32:05. I think almost all answers are clear: smaller models with good systems could be more powerful. Thank you very much, Prof. Potts.

[@vaksambath7407]

This was a great lecture! Reinforces a lot of what many have been talking about in the space, so the idea of convergent thinking aligns. Looking forward to more courses on here.

[@mikehynz]

After seeing this I can't believe that I could have just missed it. This is so important, and I am grateful to know about it now.

[@tjanardt]

A very enlightening talk by Prof.Chris . Thank you.

[@youngzproduction7498]

Bros, this change my view on the model. Your clip unlocks many new ideas for me. 🎉 great work!!

[@alirizaerfan1969]

What a great and insightful lecture! Learned a lot. Thanks much.

[@asishjoshi5774]

A master piece of explanations connecting the dots! ❤

[@lLvupKitchen]

Great lecture. Really helped me crystalize the concept of thinking in system

[@AndreiLop]

I loved this talk. Very close to my experience building AI based systems in Google, Apple and other companies. "The engine" (AI/ML/NLP) is very important. but the whole performance will be driven by the system ("formula 1" car) as a whole. Many important topics about the system design by the author.

[@Neura1net]

The Sclar paper blew my mind. Thank you for this great talk.

[@teacherdavidictcomputersci9737]

Its always been compound system, not just LLMs , it's kind of common sense coming from a computer science background. Great video thanks.

[@saurabhsrivastava69]

Standford lectures never disappoints in content

[@potsy555]

in *context

[@juanantonionavarrojimenez2966]

Thank you for put El Quijote in your wall. Best regards from Spain.

[@meisherenow]

Old-fashioned software: control, predictability, testability. LLLs: power, flexibility, generality. Together: controllable, testable, predictable, flexible, general power

[@vbridgesruiz-phd]

I am a fan of this approach for prompting with dspy because it encourages users to think mechanistically about the parts of their prompts. With larger context windows, we can fit more parts!

[@aproperhooligan5950]

Fantastic discussion and reality check. Keep it coming, please.

[@bdeceulaer]

Great lecture: thank you!

[@simonthompson1099]

Great lecture - thank you. On DsPy, I've done a lot of investigation and my conclusion is that there is a massive problem with it. If you have a small, toy problem it makes sense. The optimisation problem over a data set of say 20 or 30 possible 1 shots is obviously fine... but the promise is that you could create n-shot prompts with it, and reasonably you will be searching over 100's or 1000's of possible prompt candidates. At that point the optimisation is more or less dead in the water because it doesn't seem that a tree search works and you're just looking over a combinatorial space.

[@null4624]

Learned a lot, thank you.

[@dewinmoonl]

chris potts is always a good watch : )

[@IsabellaMoore-n7f]

This was great!

[@learnbydoingwithsteven]

However researchers advance in this field of language processing/inderstanding, one aspect is risk control on “random walk.” On system outputs, the other is on input interpretation. It seems to me that these engineering aspects could be incorporated in the models, with innovative designs in the future.

[@learnbydoingwithsteven]

Very insightful.

[@shivibhatia1613]

Brilliant questions and explanations

[@BitShifting-h3q]

thank you !! so happy to have found this

[@diga4696]

Fascinating talk! While I agree that compound systems are critical, I wonder if the future of AI might involve a unification of models and systems, where the 'peripherals' evolve into integral modalities of the architecture itself. Couldn’t these compound systems eventually become emergent behaviors within a truly scalable architecture? That said, rather than focusing on system design as an external layer, wouldn’t it be more impactful to explore architectural innovations like active inference or test-time adaptations to improve generalization and scalability? For instance, refining pre/post-training processes could allow for more dynamic integration of tools and capabilities, effectively bridging the gap between model and system. In my view, attention-based architectures still hold a decisive edge over external system optimizations-but perhaps the two approaches are not mutually exclusive. Manning’s vision of coordination between smaller, specialized models and tools does suggest a fascinating synergy between attention mechanisms and compound systems.

[@Maximos80]

I like the way you think. Interested to discuss this further with you.

[@danielnofal]

Completely agree that the model is just a part of it and we should be talking about compound systems. I would say that probably WE are a compound systems of neural networks competing for control pretty pretty pretty much like inside out movie.

[@IsxaaqAcademy]

Great perspective

[@cmdrblahdee]

My thinking is that a truly robust AI model would "spin off" new small models to perform certain tasks, and evolve them with machine learning. If it turns out to be a good evolution that's used multiple times, it would hold onto it. Otherwise, it would let it go and create a new one if it ever needs to do that task again. The benefit of this is less "hallucinations" because each model it creates is focused on a particular goal. Also, as better trained/designed models are created/discovered, updates can be pushed to the AI that only affect that specific component. So, for example, generating these micromodels and evolving them quickly to do the task is something the AI would need to be very efficient at. Communicating with the user is also something it will need to be good at. Updating the micromodel wont effect how it communicates with the user. Additionally, it might get to a point where, say, it has a communication micromodel, but it creates a sub-micromodel for dealing with a specific user.

[@RanjeetKumar-ql9by]

Enlightening talk!

[@Trnd-Labs]

The literal only way to really get an understanding of what's possible at the current moment is to just find a simple project and dive in. Like the speaker said, we seem to forget the core best practices. The only way to understand the capabilities or how to leverage them is to just start building.

[@eTeecha]

Neither system is better in terms of reliability. The answer states "Neither is better". The more dangerous option is the "Second (10B parameter LLM with web access)". The preferred option is the "Second (small model operating locally using chat history)". The expected development in 2026 is "Second (systems consisting of multiple models and tools)".

[@juanantonionavarrojimenez2966]

Wonderful video.

[@yumingliu7403]

This is a brand new architecture of the AI system that may be able to make large impact on people's life, we currently have many large language models, GPT, Gemini, LLaMA, etc., if they can be combined and interacted with each other, there might be a great chance to build more and more inteliggent AI system. Now the question is, how can we, as a developer for example, get start with developing a system like this, are there any resources, opensource project, tutorials or guidlines to follow, thanks.

[@samyio4256]

Thank you so much! Got it :)

[@NLPprompter]

22:03 is that prompt for RAG that's seems sucks prompt...

[@DistortedV12]

37:05 most important part.

[@ionuchin]

Interesting... how about writing prompts in JSON format (not output, but input)? It gives some advantages for prompt generation.

[@VKjkd]

Oh wow. I didn’t think of this. Any examples of what/why this is useful? I can imagine it reduces issues in attention.

[@sitedev]

I think that’s pretty much what JSON mode and function calling achieves already.

[@Epistemophilos]

Nice talk. It annoys me, I must admit, that 'learning' is misused in LLM space. There is no learning (in the ML) sense here - it's just picking an input for a regression model that corresponds to the kind of output you desire.

[@statebased]

Might the choice of presenting models as abstractions, and systems as "something else", be a communication strategy, just like the choice to emphasize models?

[@LatentSpaceD-g3p]

love the wheels on the f1 engine- they appear to be 2 skateboard wheels and 1 roller-blade wheel !!

[@ceilingfun2182]

It’s been known since GPT-2 that one prompt doesn’t work the same way on another model.

[@justwanderin847]

where are you on discord?

[@AlgoNudger]

Thanks.

[@JohanZahri]

There's gut brain and there's head brain; so far we have gut brain in gpt models; how do we translate the head brain mechanics in our system?

[@mmasa1]

is compound AI system another word or way to describe AI Agents?

[@MegaStatis]

Enlighten talk.

[@richardnunziata3221]

It would be nice if we started using blockchain and cryptography so access and be controlled to critical resources permissions and monitoring of excess accumulations of access keys in the blockchain by any one system

[@GrowStackAi]

Don’t call it a robot; it’s an intelligent assistant 🔥

[@box4soumendu4ever]

...thank you SIR... got the key, its multiplayer systems we need to focus more... and the marketing side of wisdom will always be behind academic side, selling other's original contribution 🙏🙏👏👏👏👏

[@DistortedV12]

Define compound systems? Can someone help me skip to that point to save time

[@reluminopraha5948]

Any system which uses more parts then one LLM (of whatever size) in order to provide better results and/or for less money.

[@yannickpezeu3419]

Don't we focus on the model because it is the hardest part to build ? Prompts ans sampling strategy can be handled by any good engineer within a month at most.

[@y0k0z00na]

QQ: when will OpenAI be forced to change their name?😅

[@neoaistudios]

5:50 in my case is converged thinking, Im creating this system to create infinite diverse content/films, thought oromots

[@balapillai9511]

Hey Stanford, the thought that comes to mind when people like me benefit from such thesis mostly composed and shared by the US institutes enabled by government budget directly and indirectly orchestrating the other government institutions to collaborate to enable research then in the backdrop of development by nation states who copying made it to number 2 threaten and ally against must be shown their place in democracy by building statistics around how their own are engaged at grassroot and what their actions imply in contradiction to their glorification. this for now. shall bb soon in mainstream.

[@RussoConcerned]

If a publicly traded company chooses to outsource overseas to cut costs, it should start by outsourcing top-tier management positions like the CEO, CFO, and CTO. By saving millions paid to these executives, the company could preserve the jobs of many American workers, who, when gainfully employed, would contribute significantly more to the local economy than a small group of overpaid executives. Why is it acceptable to outsource someone else’s job but not theirs? For every CEO outsourced, the company could retain 10 employees whose spending on homes, goods, and services would have a far greater positive impact on the community. A CEO isn’t going to buy 10 homes or spend 10 times as much locally, so why not apply the same cost-cutting logic to their roles? Selectively protecting executives while sacrificing workers only exacerbates inequality and harms the economy in the long run. H1B program overall is good. There is an abuse component to it. However,outsource of the services sector should curtailed. When Elon got his H1B, the scale of outsourcing we know of today did not exit.

[@nicholasbailey6236]

The regulation of models is one of the dumbest things to come out of the AI revolution and it reminds me of how it used to be a crime to mail a copy of Schneier's "Applied Cryptography" book outside of the US.

[@kasgol-zl9xo]

I am not quite sure this system approach will take us back a step into machine paradigm, and comparing it to a car might not be the right analogy.

[@TheGenerationGapPodcast]

I have being saying that for years. not just systems but intelligent system. Basic systems are just input output. It more like cybernetics like systems.

[@viky2002]

he sounds like hulk

[@ProgrammingWIthRiley]

smart smart smart

[@Bonhocon]

For a 1 hour long talk, it’s really too bad that you didn’t slice it into chapters, not even tried to clearly define your idea of compound system vs more specific forms, for ex. CoT or web of experts. Of course everything is a system and the success of AI depends not just on the model but also heavily on the others parts: post training alignment, prompt engineering, sampling and quality of the input materials etc. All is known and kind of obvious. So I think it deserves a clearer differentiation to add more values for the audience.

[@Trnd-Labs]

Were literally one upgrade away from entering a whole new world. When o1-preview/mini gets tool calling and multimodal functionality boy oh boy will be off to the races.

[@saadowain3511]

What does he mran by compund system.. agents!?

[@ArgoCrawler]

I predict the winners in this new world will be those that most efficiently work with their ai tool(s). Obviously, redundancies should be removed for more efficency.

[@justwanderin847]

We do NOT need government to regulate AI

[@TheLiteralist-j5h]

nor companies

[@dadsonworldwide3238]

You did Reagan thatcher traded & negotiated it far from American domestic courts jurisdiction and workers to tiawan for 50 yrs. Even before that people started world Wars to stop usa from texting this 2024 reply in concert with TV radio and we did 80+ yrs of ease of access teaching men and myth in stead .

[@dadsonworldwide3238]

The question is why did it have to be government held far away during the nuclear age and could we have not allowed those blocked out of echo chamber feilds miss aligned learning the hardway and instead allowed those who didn't need 1945s Smith_mundt act to get along the ability to live life out building there multi generational project where they left off puritanizing English and pilgrimage to confirm it common sense objectivism to program it with. We didn't have to wade through so many bad idealogy to now seemingly be left to fight back through it anyway for true optimization

[@dadsonworldwide3238]

If your American your very wall outlet y axis plugs +/- grounded by planetary nature dictating phase changes is just how fine tuned this is encoded every step of traceability has been planned for. Lol But here at the finish Line over since ww2 it's out of context problems are still present and not dealt accordingly

[@Crawdaddy_Ro]

Then how should it be regulated?

[@joe_hoeller_chicago]

Well, I built this 5y ago and got cancelled for it. 🤷‍♂️ I find it funny how Stanford vc’s are saying this now that they funded a company that just launched this. I’ve used this system my whole career since. This is nothing new.

[@MohdAli-nz4yi]

Disappointing talk. Everyone and their grandma knew LLM's would just be a component of the system, whoop de doo. Also the bitter lesson.

[@SubhamKumar-eg1pw]

Thinking about this…

[@danielvalentine132]

Beyond the click bait, and the first 10 minutes, the rest of the video is a good demonstration and explanation of good systems thinking and architecture. There is a enough dogma around “prompt engineering” and “models”. Not only did he remind us that systems are definitely the way, but he also points us in the right direction. I thought this was a very rewarding talk if you listen to the whole thing.

[@MohdAli-nz4yi]

@@danielvalentine132 I watched the whole thing. I appreciate Stanford sharing this content, I am very grateful for that. But this talk truly did not contain anything new or interesting in my opinion. Especially the "some questions to mull over" section. That pissed me off a bit actually. "Which is more reliable? A giant LLM that embeds a snapshot of the entire web as of today or A tiny LLM working with an up to date web search engine" He asks leading questions, makes false dichotomies. Especially trying to take credit "for predicting" that LLM's are used as components to software systems. This sort of corporate fluff talk BS is not what I'd expect from a university professor.

[@user-sn4sg2zx7g]

You saved me a bunch of time, thank you

[@skane3109]

This lecture was, for me, excellent. It gave highly informative insights into AI development concepts as of late 2024. It also referred many resource recommendations to those people who feel they are beyond the simple concepts discussed. (That’s not me). Kudos to everyone that shares helpful information to the world at large. We should resist the temptation to bash the contributors and instead offer our own ideas, solutions, … or our silent humility.

[@AppointmentWithJase]

AI is the worst thing humanity has invented since nuclear weapons.