AI Hardware, Explained.

[@a16z]

For a sneak peek into part 2 and 3, they're already live on our podcast feed! Animated explainers coming soon. a16z.simplecast.com/

[@cmichael981]

doesn't look like part 2/3 are up on the podcast feed (anymore at least) - any chance those video explainers are coming out still?

[@a16z]

Timestamps: 00:00 - AI terminology and technology 03:54 - Chips, semiconductors, servers, and compute 05:07 - CPUs vs GPUs 06:16 - Future architecture and performance 07:12 -The hardware ecosystem 09:20 - Software optimizations 11:45 -What do we expect for the future? 14:25 - Sneak peek into the series

[@jack_fischer]

The music is very distracting. Please tone down in the future

[@Inclinant]

In the usual case of floating-point numbers being represented at 32-bit, is this why quantization for LLM models can be so much smaller at around 4-bit for ExLlama and making it so much easier to fit models inside the lower amounts of VRAM that consumer GPUs have? Incredible video, interviewer ask really though provoking and relevant questions while the interviewee is extremely knowledgeable as well. It's broken down so well too! Also, extremely grateful to a16z for supporting the The Bloke's work in LLM quantization! High quality quantization and simplified instructions makes LLMs so much easier to use for the average joe. Thanks for creating this video.

[@msclrhd]

It's a trade-off between accuracy and space/performance (i.e. being able to fit the model on local hardware). A 1-bit number could represent (0, 1) or (0, 0.5) as it only has 2 values. With 2 bits you can store 4 values, so you could represent (0, 1, 2, 3), signed values (-2, -1, 0, 1), float between 0 and 1 (0, 0.25, 0.50, 0.75), etc. depending on the representation. The more bits you have the better the range (minimum, maximum) of values you can store, and the precision (gap or distance) between each value. Ideally you want enough bits to keep the weights of the model as close to their trained values so you don't significantly alter the behaviour of the network. Generally a quantization of 6-8 offers comparable accuracy (perplexity score) with the original, and below that you get an exponential degredation in accuracy, with below 4-bits being far worse.

[@NarsingRaoschoolknot]

Well done, very clean and clear. Love your simplicity

[@AlexHirschMusic]

This is highly informative and easy to understand. As an idiot, I really appreciate that a lot.

[@lnebres]

An excellent primer for beginners in the field.

[@Matrix1Gamer]

Guido Appenzeller is speaking my language. the lithography of chips are shrinking while consuming lots of power. Parallel computing is definitely going to be widely adopted going forward. Risc-V might replace x86 architecture.

[@TINTUHD]

Great video. Tip of the computation innovation

[@AnthatiKhasim-i1e]

"To remain competitive, large companies must integrate AI into their supply chain management, optimizing logistics, reducing costs, and minimizing waste."

[@lerwenliu9263]

Love this Channel! Could we also look at the hunger for energy consumption and the impact for climate change?

[@dinoscheidt]

1:24 Ehm… I would like to know, what camera and lens/focal length you use to match the boom arm and background bokeh so perfectly 🤐

[@StephSmithio]

I use the Sony a7iv camera with a Sony FE 35mm F1.4 lens! I should note that good lighting and painting the background dark does wonders though too

[@kymtoobe]

This is a good video.

[@IAMNOTRANA]

No wonder nvidia don't care about consumer GPU anymore.

[@stachowi]

Yup, cash grab

[@nvr1618]

Excellent video. Thank you and well done

[@Doggieluv25]

Really helpful thank you!

[@adithyan_ai]

Incredibly useful!! Thanks.

[@chenellson489]

See you at NY Tech Week

[@vai47]

Older Vox style animations FTW!

[@stachowi]

This was very good

[@MegaVin99]

Thanks for video but 4 mins before getting to any details in a 15 min video?

[@billp37abq]

This video makes clear WHY DSP [digital signal processing] chips were implementing sum{a[i]*b[i]} in hardware!

[@SynthoidSounds]

A slightly different way of looking at Moore's Law is not about being "dead", but rather becoming irrelevant. Quantum computing operates very differently than binary digital computation, it's irrelevant to compare these two separate domains in terms of "how many transistors" can fit into a 2D region of space, or a FOPS performance. Aside from extreme parallelism available in QC, the next stage from "here" is in optical computing, utilizing photons instead of electrons as the computational mechanism. Also, scalable analog computing ICs (for AI engines) are being developed (IBM for example) . . . Moore's Law isn't relevant in any of these.

[@LeveragedFinance]

Huang's law

[@shwiftymemelord261]

it would be so cool if this main speaker was a clone

[@billp37abq]

AI and cloud computing face power supply issue as cryptocurrencies? "Cryptocurrency mining, mostly for Bitcoin, draws up to 2,600 megawatts from the regional power grid-about the same as the city of Austin."

[@joshuatruong2001]

The Render network token solves this

[@LeveragedFinance]

Great job

[@thirukaruna7469]

Good one, Thx.!

[@gracekim2863]

Back to School Giveaway

[@RambleStorm]

Geforce 256 aka GeForce 1 wasn't even Nvidia's first gpu let alone the first ever PC gpu... 😅😂

[@sergiocayuqueov]

Interesting

[@antt8550]

The future

[@billp37abq]

AI power consumption has doomed it to failure before it has started? kzbin.info/www/bejne/ooPcZoavbqqfmNk

[@mr.wrongthink.1325]

The music is unnecessary and actually annoying.

[@1SlipperyPenguin]

Sounds like a bad nightclub , stop the music