GPT-4o mini Prompt Chain: Legit TRICK for DIRT CHEAP AI with SOTA Accuracy

[@HerroEverynyan]

This is why I love your stuff -> "I think one of the most important things you can do in the age of AI is make sure that your information diet is as clean as possible. There's going to be so much content generated; there already is so much content you feel on a daily basis. You know what this is like. I think it's important to build and use tools that help you filter out the noise and keep yourself in a low-noise, focused environment."

[@jaysonp9426]

I'm so glad you make videos. People should be freaking out about this but all I hear from most channels is how "AI is slowing down"

[@ronilevarez901]

Maybe because most people who could be interested in the new model can actually pay for the old one and they are already paying for it so this, while good, is just a small improvement on their finances.

[@indydevdan]

tyty 🙏 and I completely agree. There's so much potential in this technology we're just cracking the surface. "AI is slowing down" is garbage clickbait imo.

[@bradlegassick9327]

More on prompt chaining & fusion chains please! Such good info. Thank you

[@AgentZeroFW]

I love 4o-mini, now I use it exclusively in Agent Zero.

[@MontyCraig]

This is such a game changer! Plus gpt4-0mini is free in cursor.

[@bluebabboon]

Awesome content as always

[@WhenHopeWhispers]

Well done, this is actually genius especially for how simple it is. 👏👏👏

[@milev051]

I was just looking at your video from 2 weeks ago about sonnet and I thought, this was before GPT-4 Mini, now it will be much cheaper. And the next thing I saw was this video xd

[@bukitsorrento]

Shut up and take my money 😂 Would love to add OpenAI Batch Job for the fusion/scheduled workflow to push the cost down even more, like 50%. Thank you for making these videos; I really appreciate it.

[@bukitsorrento]

KZbin channels are also a source of signals. I receive signals from your yt channel, not your blog :)

[@forwatching2310]

Hey, loved your video - super valuable info & you seem like the rare person solving problems people don't realise exist yet. Just a question - how does Autogen relate to prompt/fusion chaining?

[@forwatching2310]

hey all good - came across your previous fusion video where you talk about it

[@mr2octavio]

Thank you for your content!

[@93cutty]

I always look forward to your videos

[@Drone256]

Your chains and evaluator requiring 10 queries total cost 60% of the price of just using GPT-4o alone. I am assuming your evaluator is a single query and I’m assuming all queries have the same average input/output tokens as a single query to GPT-4o. Soooo…. Are the results better at 60% of the cost?

[@indydevdan]

Not sure where the 60% is coming from. With 10 prompts it would be 30% since gpt-4o mini cost = $0.15/per million and gpt-4o cost = $5.00/per million. I think you're missing the part where ~30 gpt-4o mini prompts equals the cost of a SINGLE gpt-4o prompt. In this video we use 8 gpt-4o mini prompts in total (We ran 4 gpt-4o mini prompt chains with length 2 = 8). The evaluator does not run a prompt it combines the results from the prompt chains. GPT-4o mini is ~30x cheaper. That means I can literally run 30 prompts of the same size and the cost would equal a single GPT-4o prompt. Are the results better with prompt chain of length 30? Yes, thankfully that's WAYY overkill and GPT-4o mini prompt chains and fusion chains perform better (or even) with less than 10 total prompts. Just 10 prompts. It takes 30 gpt-4o mini prompts to equal 1 gpt4-o prompt cost wise.

[@Drone256]

@@indydevdan I completely understand. My number comes from a variation of your example and the pricing as I read it at OpenAI’s website. You are on the right track. There are endless ways to mix this up to get improved results. Think bigger.

[@pawsjaws]

@@indydevdan might be a bit late, but have you tested these on Benchmarks? Or is it just intuition?

[@fkxfkx]

do you make any distinction between prompts and queries? you discuss prompt chains and fusion but is there some equivalent for queries? query chains and fusion? or do prompts take the full role of both prompt and query?

[@mikecourian]

Curious how you are getting Token calculations in the bottom right of your Cursor window 🤔?

[@indydevdan]

tokenizer vs code plugin (rough estimate, models use different tokenizers as you likely know): marketplace.visualstudio.com/items?itemName=marxism.ai-token-count

[@lakergreat1]

how could you implement this with local models?

[@ytubeanon]

I am a huge fan of 4o-mini for its effectiveness vs cost. What if I wanted to use prompt chaining to make a chrome addon, would I modify the 'Minimal Prompt Chainables' gist and use that? Or is there a more abstract, universal kind of template

[@indydevdan]

Same here - 4o-mini let's you do so much for so little. Yes, you can modify Minimal Prompt Chainable to your use case.

[@reinaldosotillo4890]

This person does know how to use LLMS. Could you talk about fine tuning to gpt4 mini?

[@theraybae]

Where can I try out the RecommendWorkflow with agentics?

[@junmagic8847]

Will you open source the zero noise streamlit app?

[@internetperson2]

Is Zero Noise your own tool ? Is it open source ?

[@masterapofis4997]

They should have warned us that GPT-4o mini only has 50 questions every 4 hours at this rate. Does it force you to have multiple accounts or have to share it, ceasing to be free, and when does that happen? We will go to another AI. We want GPT-3.5 back!

[@insight-guy]

Amazing. Can you make a Google Collab?

[@RyanJohnson]

1:53 "By only a drop of 10% you have a gain of 30,000%" how does the math work out to this?

[@indydevdan]

At worst, gpt-4o mini performs ~10% worse than gpt-4o. At the same time it is 30x cheaper. 30x = 30,000%.

[@lydedreamoz]

So... I'll get the same results as a state-of-the-art model, but it will take 10 times as long...?

[@eintyp4389]

gpt-4o-mini is quicker than gpt-4o by a lot and you can just run all the querys in parallel (I mean the 4 chains. You oviousely need to still feed the output into the next part of the chain for each one but thats just 4 querys with a realy fast model). So no it will probably even be quicker and whats more relevant its avordable while the quality remains the same.

[@nedkelly3610]

I long suspected that evaluator functions with parallel chains is key to accurate llm outputs.

[@indydevdan]

100% - code is still essential while llm accuracy increases over time. I do think long term, most evaluators will just be another prompt (or chain) that validates the output.

[@FasutonemuMyoji]

I wonder if your prompts would work better or worse if you spelled the word "scraped" correctly in the prompt. I hope it does not think you are getting the website results from the scrap bin 🚮

[@gtrguy17]

@indydevdan does aider integrate with gpt4o mini?

[@indydevdan]

Yes!