Dylan Patel-GPU Shortage, Nvidia, Semiconductor Supply Chain

Dylan Patel is Chief Analyst at SemiAnalysis a boutique semiconductor research and consulting firm specializing in the semiconductor supply chain from chemical inputs to fabs to design IP and strategy. The SemiAnalysis substack has ~50,000 subscribers and is the second biggest tech substack in the world. In this interview we discuss the current GPU shortage, why hardware is a multi-month process, the deep learning hardware supply chain and Nvidia's strategy.
SemiAnalysis: www.semianalysis.com/
Dylan: / dylan522p
Patreon: / theinsideview
Outline
00:00 Highlights
00:31 Dylan's Background, How Semi Analysis Started
01:34 What Caused The GPU Shortage
02:42 Hardware Involves Multi-Months Processes, Even Years
03:20 Six Months Between The Order And The Datacenter Installation, Different From Open Source
04:25 Putting Chips Together With The Memory Is TSMC's Bottleneck
05:20 A Humonguous Increase In GPU Capacity, Comparing GPU Clusters
06:10 Five To Seven Companies Will Have GPT-4 Compute In The Next Year
07:39 Manufacturing Won't Be The Bottleneck Even If Scale Is All We Need
08:46 125 thousand H100s For Training In The Next Year Or Two
09:39 The Nvidia Customer Democracy, Independent Of The Demand
10:51 Why Nvidia Invested In Inflection
11:27 Final Thoughts On Semiconductors
Patreon supporters:
- Tassilo Neubauer
- MonikerEpsilon
- Alexey Malafeev
- Jack Seroy
- JJ Hepburn
- Max Chiswick
- William Freire
- Edward Huff
- Gunnar Höglund
- Ryan Coppolo
- Cameron Holmes
- Emil Wallner
- Jesse Hoogland
- Jacques Thibodeau
- Vincent Weisser

[@TheInsideView]

Outline 00:31 Dylan's Background, How Semi Analysis Started 01:34 What Caused The GPU Shortage 02:42 Hardware Involves Multi-Months Processes, Even Years 03:20 Six Months Between The Order And The Datacenter Installation, Different From Open Source 04:25 Putting Chips Together With The Memory Is TSMC's Bottleneck 05:20 A Humonguous Increase In GPU Capacity, Comparing GPU Clusters 06:10 Five To Seven Companies Will Have GPT-4 Compute In The Next Year 07:39 Manufacturing Won't Be The Bottleneck Even If Scale Is All We Need 08:46 125 thousand H100s For Training In The Next Year Or Two 09:39 The Nvidia Customer Democracy, Independent Of The Demand 10:51 Why Nvidia Invested In Inflection 11:27 Final Thoughts On Semiconductors

[@tobiaswegener1234]

The quality of this Interview was superb. I get a little chilled, when I hear how much compute is produced and distributed.

[@TheInsideView]

Transcript: theinsideview.ai/dylan

[@radicalrodriguez5912]

what did this guy study at university?

[@JazevoAudiosurf]

at this point a miracle would have to happen to prevent China-Taiwan conflict. as soon as that happens hardware does become the bottleneck

[@Extys]

Not really? TSMC is 1-2 years ahead of the competition. This would only slow progress by that much, about the same as the COVID chip shortage in 2020-2021.

[@JazevoAudiosurf]

@@Extys and how does that contradict what I'm saying?

[@Extys]

@@JazevoAudiosurf You imagine a bottleneck