Optimization of LLM Systems with DSPy and LangChain/LangSmith

The most interesting LLM applications are less a single LLM call and more a complex system that relies on multiple LLM calls. Prompt engineering all these LLM calls explicitly can be painful and time consuming.
DSPy pioneered the idea of "optimization" of LLM systems; using automatic evaluation to optimize the instructions, few shot examples, and even the weights.
LangChain and LangSmith are starting to incorporate some of these in an *online* manner.
This video covers why you should think in terms of LLM systems, why optimization can be useful, what components make the most sense to optimize, and how to use DSPy and LangChain/LangSmith to do so.

[@cm_dev]

I appreciate your commitment to posting relevant and informative content on a regular basis!

[@istvandarvas3372]

This was so awesome I cannot find the words to describe it. THANKS! Finally, I'm starting to feel like we're heading in the right direction with this DSPY ;) We're beginning to reach a point where AI, serving as an engine, can now be utilized alongside proven, well-developed software engineering principles. These principles are genuinely advanced, underpinned by 50 years of industry experience.

[@nqnam12345]

Thanks ! Great , I always had trouble with prompting

[@robboerman9378]

Very informative, thanks for sharing. Is there anywhere I can watch the talk that the slides were originally used for?

[@attilavass6935]

This is great, but my pain is how to use DSPy without going bankrupt :)

[@sammathew535]

It does beautiful caching across the system, so it only calls the backend LLM, if you haven't done so, until then!

[@ppbroAI]

Can we use gemini as a bigger model to optimize lesser models?. I mean gemini API is free and its pretty capable

[@diallo9149]

very informative , I'll definitely look into it ✌️✌️

[@codekiln]

World love the slides link!

[@waynehawley814]

Normalize the Audio Next time please... i have to increase the volume by 500%...

[@fraternitas5117]

why would I use langchain if I had this?

[@miguelovallevillamil4953]

IMO and from an application-building perspective, LangChain is necessary since you cant give the final user of your application a DSPy code as frontend. In the video they mention the idea of using DSPy with LangGraph, which I think resonates with what I said preivously ~ rather than seing them as a replacement for each other, they can be used together to build your apps.

[@istvandarvas3372]

I think you need both! The small discussion at the end about Keras which is on top of TensorFlow illustrates that the same idea can be integrated. We need 'glue' and we also need higher-level abstraction building blocks for 95% of applications out there. The mariage of LangChain and DSPy fall into this category, in my opinion. Therefore, I am looking forward to learning more about this and am excited to use these higher-level building blocks in a LangChain application. It will speed up development and testing, and at the end of the day, that is what matters most!

[@Karl-Asger]

I'm wondering the same thing. I only briefly used LangChain, but have bounced around all the different frameworks since then and have been using DSPy for a while now. While I see what Miguel and Vandarvas are saying here, I don't understand what Langchain can do that DSPy can't. The DSPy abstractions are much more peaceful to work with and allow great flexibility. "LangChain is necessary since you cant give the final user of your application a DSPy code as frontend" - not sure if I'm misunderstanding, but when would you give LLM app code as a frontend? Interested to hear more on this. It sounded like Harrison was really pushing the idea of going between LangChain and DSPy, and of course pushing LangSmith as the product which looks very nice I have to say.

[@haralc6196]

Am I the only one who things that DSPy only sounds nice in concept and probably only work on 3% on all possible GPT use cases, and for the other 97% it's a complete BS?

[@marcusandatlas844]

DSPy is pretty cool but Omar *really* needs to work on his communication and presentation skills.

[@ivant_true]

it is cool but naming is absolutely terrible!

[@maksimhajiyev7857]

LangChain and the overall GEN AI is a huge hype , people will get back on track soon. These technology is not stable with huge overhead. It should have been done in low level lang. The problem with AI "revolution" it is dominated with PHDs and data analysts who never did real enterprise soft