NEW TextGrad by Stanford: Better than DSPy

[@dennisestenson7820]

This is a concept I'd been considering myself, but I never thought of it as autodifferentiated text. Fantastic that research is being done in this direction. I knew it'd be a good idea.

[@Caellyan]

I criticized this has to be done manually, but never thought of chaining 2 LLMs to achieve it. Though, it does make getting slightly better answers 3x more expensive. I guess it's useful for unsupervised learning though.

[@kenchang3456]

Thanks for the video. I missed the boat with DSPy but it's good to know you can just go ahead with TextGrad.

[@giladmorad4348]

Thanks for the video, it’s very insightful! I have 1 thought: 1. Textgrad and DSPy can be combined. As DSPy is mostly based on ICL and this framework focuses more on signature optimization. Additionally, the researchers in Stanford mentioned that the combined prompt on one occasion improved the prompt by 1% and it should be further studied.

[@matty-oz6yd]

DSPy is ICL and prompt optimisation combined. I hope they add text grad in somehow though

[@giladmorad4348]

@@matty-oz6yd yea, good correction. I hope they add Textgrad in as an optimizer.

[@hussainshaik4390]

Their mipro v2 optmizes literally doing the same

[@fingerstyleguitarjustingao729]

great video, hope for you more advanced explain and experience on TextGrad!

[@mydetlef]

OK, I'm a n00b. But why should I use two models when the smarter one can give me the optimal answer straight away? In which scenarios do I need all these expensive iterations? Will I then have predefined prompts for recurring queries of the same type that can be answered directly on my smartphone by a small model?

[@brandonheaton6197]

Solid. I knew if the guy behind DSPy could build that, there was a better version imminent

[@jmanhype1]

sounds like we need a middleware complexity assesor that can sit in the middle and auto reject if it doesnt meet that balance

[@matterhart]

Thanks stanford, though I would have called it backpromptigation. ;)

[@mlcat]

26:51 what does 0 demonstrations mean? No examples of good output, only original prompt?

[@mydetlef]

Answer from Copilot: Yes

[@DannyGerst]

You said that you used in on your tasks. Can you release part of that code in the wild? It would be really great to see a live example. That was the thing I found very challenging with DSPy. Only with the storm project I started understanding how it should work ;-)

[@code4AI]

Start with the four Jupyter Notebooks that I provided and you will see that you have immediately multiple new ideas for your specific tasks. I plan a new video on my insights, given my testing and maybe I have an idea how to optimize the TextGrad method further ....

[@pensiveintrovert4318]

It is 3 months later, has either of the two approaches proven to be practically useful and is being used today?

[@pensiveintrovert4318]

How is this different from prompt tuning (not engineering)?

[@code4AI]

Explained in the video.

[@Anonymous-lw1zy]

Superb explanation! Thank you!

[@hoomansedghamiz2288]

Here's an unpopular opinion: could this be considered a misuse of the notation for auto-differentiation and backpropagation? For any graph to be differentiable, it must be acyclic-like a Directed Acyclic Graph (DAG), which is typical for neural networks. However, in the LLM sphere, we see pipelines incorporating cycles, such as the RAG where blocks are repeatedly cycled through, forming what might be described as Directed Cyclic Graphs (DCGs). While using PyTorch's clean and modular syntax is appealing, applying auto-differentiation in this context could be seen as a stretch (personal opinion).

[@MindEmbedding]

Thanks for another great video! I like your presentation style. What kind of software do you use for your slides?

[@asadad5162]

Great video, very informative. Textual Gradient is such a pretentious concept for me, but I do look forward to try TextGrad out. At least it is a systematic method to perform prompt optimization.....

[@artur50]

Thanks for the links to colabs…

[@stephanembatchou5300]

Very informative. Thanks

[@GeoffLadwig]

Great! Thanks

[@whig01]

Seems like one can prompt optimize for the same level system and never lack coherence.

[@spkgyk]

Amazing video! But pseudo as in pseudo-code is pronounced like sudo (syuudo) Not smart enough to correct anything else in this video lmao, keep up the good work! Love the channel