AI Scaling Walls

This week we talk about neural networks, AGI, and scaling laws.
We also discuss training data, user acquisition, and energy consumption.
Recommended Book: Through the Grapevine (amzn.to/3ZazzfH) by Taylor N. Carlson
Transcript
Depending on whose numbers you use, and which industries and types of investment those numbers include, the global AI industry-that is, the industry focused on producing and selling artificial intelligence-based tools-is valued at something like a fifth to a quarter of a trillion dollars, as of halfway through 2024, and is expected to grow to several times that over the next handful of years, that estimate ranging from two or three times, to upward of ten or twenty-times the current value-again, depending on what numbers you track and how you extrapolate outward from those numbers.
That existing valuation, and that projected (or in some cases, hoped-for growth) is predicated in part on the explosive success of this industry, already.
It went from around $10 billion in global annual revenue in 2018 to nearly $100 billion in global revenue in 2024, and the big players in this space-among them OpenAI, which kicked off the most recent AI-related race, the one focusing on large-language models, or LLMs, when it released its ChatGPT tool at the tail-end of 2022-have been attracting customers at a remarkable rate, OpenAI hitting a million users in just five days, and pulling in more than 100 million monthly users by early 2023; a rate of customer acquisition that broke all sorts of records.
This industry’s compound annual growth rate is approaching 40%, and is expected to maintain a rate of something like 37% through 2030, which basically means it has a highly desirable rate of return on investment, especially compared to other potential investment targets.
And the market itself, separate from the income derived from that market, is expected to grow astonishingly fast due to the wide variety of applications that’re being found for AI tools; that market expanded by something like 50% year over year for the past five years, and is anticipated to continue growing by about 25% for at least the next several years, as more entities incorporate these tools into their setups, and as more, and more powerful tools are developed.
All of which paints a pretty flowery picture for AI-based tools, which justifies, in the minds of some analysts, at least, the high valuations many AI companies are receiving: just like many other types of tech companies, like social networks, crypto startups, and until recently at least, metaverse-oriented entities, AI companies are valued primarily based on their future potential outcomes, not what they’re doing today.
So while many such companies are already showing impressive numbers, their numbers five and ten years from now could be even higher, perhaps ridiculously so, if some predictions about their utility and use come to fruition, and that’s a big part of why their valuations are so astronomical compared to their current performance metrics.
The idea, then, is that basically every company on the planet, not to mention governments and militaries and other agencies and organizations will be able to amp-up their offerings, and deploy entirely new ones, saving all kinds of money while producing more of whatever it is they produce, by using these AI tools. And that could mean this becomes the industry to replace all other industries, or bare-minimum upon which all other industries become reliant; a bit like power companies, or increasingly, those that build and operate data centers.
There’s a burgeoning counter-narrative to this narrative, though, that suggests we might soon run into a wall with all of this, and that, consequently, some of these expectations, and thus, these future-facing valuations, might not be as solid as many players in this space hope or expect.
And that’s what I’d like to talk about today: AI scaling walls-what they are, and what they might mean for this industry, and all those other industries and entities that it touches.
-
In the world of artificial intelligence, artificial general intelligence, or AGI, is considered by many to be the ultimate end-goal of all the investment and application in and of these systems that we’re doing today.
The specifics of what AGI means varies based on who you talk to, but the idea is that an artificial general intelligence would be “generally” smart and capable in the same, or in a similar way, to human beings: not just great at doing math and not just great at folding proteins, or folding clothes, but pretty solid at most things, and trainable to be decent, or better than decent at potentially everything.
If you could develop such a model, that would allow you, in theory, to push humans out of the loop for just about every job: an AI bot could work the cash register at the grocery store, could drive all the taxis, and could do all of our astronomy research, to name just a few of the great many jobs...