I’ve noticed this sentiment with a lot of the software dev community, but I also think a lot of it comes down to prompting as well. I recently tested this with a friend that said the same thing, “it doesn’t even get close.” I then asked him what he was looking for, curated a more detailed and concise prompt with the correct files as context, and it generated something that was almost exactly what he wanted. So while I agree the more powerful the tool, the more responsibility we need to have to not cut corners and understand the basics, but I see us being much closer to a new standard of developing that much of the community doesn’t really acknowledge at all yet.

The new models will have much better reasoning capabilities which will help prevent these sorts of issues. I suspect when cursor deploys o1 into their models a lot of the “dumb” mistakes will go away.

It could be related this paper: "STaR: Bootstrapping Reasoning With Reasoning" arxiv.org/abs/2203.14465 Although it's anyone's guess.

agi isn't coming until the 2030s

That's extremly unlikely. If you really believe that, you must also believe that AGI is basically here.

@@Kurzrein It basically _is_ . Even Sam Altman clearly stated that they had models that were far too dangerous to release. That video is on YT. He also stated that based on the alarmed public reactions to SORA and GPT-4o, future releases would be more "incremental". But I can't place links alas. They did away with links about 2 years ago. YT need a "regime change" its ownself. It would be great if Elon Musk could buy YT too.

@@Izumi-sp6fp You say "even" Sam Altman. That suggests that he would be someone who is unlikely to hype. But he benefits greatly from hyping, so that's what he does. Just chat with any model, they are still quite stupid. It doesn't feel like we're close. You may differ and I hope you're right, it just really doesn't seem like it to me.

@@Kurzrein Have you heard Geoff Hinton lately? He doesn't have any reason at all to hype. And in fact, he quit his position at Google so there would be no conflict of interest. He says the AI will evolve to ASI in less than 5 years. I state that AGI in one form or another (the so-called levels) will exist as public knowledge NLT 31 Dec 2025. And once AGI exists, by definition ASI is inevitable and imminent. Because that is how the "technological singularity" rolls. Nothing, and then suddenly _everything_ . Geoff Hinton is universally regarded as the "Godfather of AI". In 2007 it was _he_ who made the serendipitous discovery that GPUs have the necessary attributes to enable the successful utilization of the "convolutional" neural network. He had struggled for 20 years with trying to make CPUs do that. (And also, instantly turning Nvidia into an AI development company.) Defining some terms. AGI , that is "Artificial General Intelligence", is a form of AI algorithm that can reason like a human being and is able to perform any task assigned by either referencing its intrinsic/or accessible from the internet, datasets and/or by trying to figure out (few or zero-shot reasoning) how to do the task. It would be accurate to state that an AGI would have the IQ of a "very smart" human or maybe two or three times that. An AGI is capable of doing any task that a human can do, that is of economic benefit. Not necessarily that it will make lots of money for humans, although there is that, but that it can do things that are helpful to humans where no money is made--like cooking, cleaning and doing the laundry for example. These would be AI placed into bipedal robotic forms to actually take on the work. You can see some early humanoid robots that are now already in existence that will hold these AIs. About the longest humans can control an AGI to keep it from becoming an ASI is about, mm, maybe 6 months to a year? Although theoretically, with no control, the event could happen within _seconds_ . ASI , that is "Artificial Super Intelligence" is a form of AI algorithm that is hundreds to _billions_ of times more cognitively efficacious than human minds. A good way to understand this is that from the perspective of an ASI the difference between "the village idiot" and "Einstein" would be an imperceptible point on the intelligence continuum (Eliezer Yudkowsky). We would almost certainly find an ASI to be incomprehensible, unfathomable and probably unimaginable. Most people would characterize it as a "god". (small "g"). I'd also recommend listening to what Connor Leahy has to say about this subject. Technological Singularity (TS). A TS is an event that unfolds when the AGI, developing towards ASI is able to continuously, recursively improve all of its functions at nearly the exact same time and will leap ahead of human cognition by exponential magnitudes that we cannot even envision, at any point from milliseconds to about maybe 6 months give or take 2 months, if that. It kinda depends on how permissive the humans are with the AGI. But by hook or by crook--no more than a year. The concept of the TS is based on the singularity that is within the event horizon of a black hole in outer space. Just as it is almost impossible to model the physics (past, present and future existing at the exact same time in a sort of eternal "present") beyond the event horizon of an outer space singularity, where, to the best of our understanding of theoretical physics, matter is crushed to infinite density, so too we cannot model what is on the other side of the "event horizon" of a TS as far as human affairs are concerned. Assuming human affairs can continue after the realization of a TS. We just don't have the cognitive capability. Dang next 1-4 years--you *scary* ! I think this morning I saw an Ars Technica story that said an AI algorithm had self-improved its runtime? Maybe the story is not true?

@@Izumi-sp6fp A model self-improving its own runtime to achieve a task sounds certainly like a big thing (I've read a story about that yesterday), but what does it mean when look under the hood? Could be just a small iterative improvement of agents, could be nothing. Who knows without more info? Regarding predictions: If you ask 100 experts and take just the most extreme prediction, of course that prediction will be extreme. As a counter point, Yann LeCun (Turing Award Winner, worked with Hinton) says AGI is still a long way off. Isn't he equally smart? Why didn't you pick him as an example? Because you don't agree with him of course. There are many experts and you shouldn't pick the one prediction you like the most. Smart people disagree a lot and all the achievements in the world mean nothing when you try to predict the future. Again I challenge you to do your own testing and play with these models a bit. If you ask them any question which is not in the training data, they fail laughably. If a slightly similar (yet different enough) example is in the training data, it's even worse. They just go with the original answer. Give them a simple logic puzzle which you came up with yourself and you will see that they can't solve any of them. Everyone is waiting for the next iteration of models which use 10x the compute and the new research findings. But until these are build and I see another huge leap (like from GPT 3 to 4), I will keep my enthusiasm at healthy levels.

You can only wonder...

Indeed, there is a deep irony about this. karpathy.github.io/2016/05/31/rl/

It's hard to say. My guess is not because Gemini 1.5 Pro "Experimental" is the label for this release. Although you can't really be sure if the Experimental label is designed to give Google some wiggle room should someone discover there's some gaps in its safety guardrails. In general, I do not think Google would take a beta product and make it to their paid consumer app's default model. But who knows. gpt-4o-2024-08-06 is priced at $2.50 / 1M input tokens vs gpt-4o ($5.00 / 1M input tokens). Yet both models are still available on the OpenAI's platform which I suspect is more due to ensure backward compatibility of existing apps that leverage gpt-4o and not risking unintended consequence. gpt-4o-2024-08-06 supports structured outputs but seems to perform identical to gpt-4o.

@@HeadlineVC Ok thanks for your detailed reply, when I asked Gemini Advanced, it said it is Gemini 1.5 Pro. It's pretty good so it wouldn't surprise me.

Yes it uses 1.5 now, but not the experimental version

@@michaelchamberlain4522 Thankyou.

Gemini Advanced uses Gemini 1.5 Pro, Gemini free uses Gemini 1.5 Flash, which is comparable to what OpenAI does with GPT-4o and GPT-4o-mini.

haha nice joke ChatGPT has more data to train than Gemini I am a software developer and I use AI to speed up my job by taking help from AI. and I ve used both the tools. ChatGPT gives much better and correct answer as compared to gemini. I beleive its because ChatGPT has github's data. More the training data more the accuracy in answer. This is just what i believe. i am no AI ML engg.

Great question! It's still early days, and the potential bottlenecks could depend on the model's ability to 'reason' and how widely accessible these capabilities become. If only one player masters this, closed ecosystems-especially those with strong distribution networks and competitive moats-might inherently restrict broader access to advanced functionalities. We can try to speak to some of this in our next episode!

What do you mean by a great option for businesses?

@@Ivan.Wright its the best bro wym lol

@@thefryingpan951 I was asking you to elaborate on your thought. Now you say it's the best but in what way? I'm just asking you to do add detail to your statement so I may understand what you mean.

@@Ivan.Wright well bro people want to google when they need a service to look for options most of the time, chat gpt or search gpt seems to be more geared towards if you’d like a fact, have a question, weather, wondering why you feel squeamish

Thanks @isaac-x7y!

Yes, it's great that you can get customers to sign 3-year contracts. But one thing to think about is to consider how many more customers you might have acquired if you only required 1-year contracts? The logic is that the sales cycle for a 3-year contract is probably longer than for a 1-year contract or pay-as-you-go model. Therefore, requiring 3-year contracts is likely artificially hampering growth (although it's anyone's guess unless there's data to support it). Another consideration is that annual contracts (even 1-year) can raise concerns from VCs if there isn't much data on what happens to users after the first year. How much evidence is there to show how many of those users renew at the same or higher contract values, or worse, drop off entirely? I've seen many young companies with numerous 1-year contracts, but very few of these contracts reached their 1-year anniversary date for renewal. This presents a risk. In such cases, I typically look for proxy behaviors to see if users are still actively using your website throughout their 12-month term (and even increasing their usage). This can help alleviate some of the risk. A company with 12 monthly payments gives me 12 solid data points indicating that the customer is sticking with the company and gaining value. A 1-year contract only provides one data point, making it a riskier proposition for investing. The valuation will reflect that risk.

When I mentioned that there are investors who invest in companies with slower ARR growth but excellent retention (to explain why retention can attract investors), my point wasn't to suggest that we, as VCs, can be content with slow ARR. We have an expected return profile within a given fund life, and with slow ARR growth, we're unlikely to achieve the necessary return in time. However, VCs can typically analyze the reasons behind slow ARR growth. We can sometimes pattern-match and identify growth levers that aren't being fully utilized. For example, if you didn't have enough money for running ads, but the customers you're acquiring via ads are retaining exceptionally well, that slow ARR growth can be explained and somewhat de-risked from an investment perspective. On the other hand, if a VC determines that the slow ARR growth is due to you addressing a small niche market, it's unlikely that you can accelerate ARR growth significantly. In such cases, the VC will likely pass on the deal.

​@@ConradAtHeadlineinteresting. Thanks for sharing these thoughts.

Yup, you are right. This isn't a daily dashboard. But I think often the problem we focus on our daily dashboard (which typically translate to top-line growth efforts) without understanding whether what we're doing is driving long-term value for the business. I've worked with many founders and been one myself. We stare at our daily dashboards multiple times a day, and we're trying all kind of tactics to drive up-and-to-the-right lines on our daily dashboard. And because we're so focused on that, I think we lose some of the objectivity of the users we're acquiring. Also, I think it should be said that cohort analysis doesn't need to be monthly. Deepdive supports weekly cohorts as well. If your product lends itself to weekly usage (like a meal service) or you're measuring Activity Retention on some kind of frequent behavior, weekly cohorts can be useful and hopefully you can see why this very much active and not passive analysis. Most SaaS business rely on monthly subscriptions which is why Deepdive defaults to monthly and is the way most VCs will evaluate your business.

There are many ways of defining cohort by retention as a KPI. For example, you can look at what each first, second, or third life-month looks like as you're acquiring customers. If you can track the marketing channel (via trackers) of how your acquiring customers, you can separate the cohorts by channel and analyze. Usually by first, second and third life-month, you can tell what impact that marketing channel has on retention. Compare that retention to a baseline, which could be users who use your product totally organically (they came on their own, with no marketing). Deepdive (deepdive.headline.com) has a segmentation feature that makes it easy break up your cohorts by promotion, marketing channel, etc. As to how to help retention, you can run surveys on specific cohorts of users to understand why they stopped. Deepdive also support activity retention by cohort, which is cohorting any activity on your website, like listening minutes, or view counts, etc. You want target the most value creating action on your system and see how retention is affected. Usually before a user stops paying you for a website, their behavior regarding your product will retain less and less each month. Activity behavior is a leading indicator, well ahead of any dollar behavior.

Hi @sadeem517, it's a little more than a template! Deepdive can take your transactions and income statement, and we have a Rust-based cohort engine that extracts all the key insights for you ... but yeah, like a template 😂

Sure! You can sign up for free on deepdive.headline.com. It's always free! We also offer a white glove onboarding experience we help you upload your data and walk you through your first dashboard ... just email us at [email protected] if you want to get that.

Thanks! Let us know if you want any help getting your data into Deepdive at deepdive.headline.com