A friendly introduction to distributed training (ML Tech Talks)

Google Cloud Developer Advocate Nikita Namjoshi introduces how distributed training models can dramatically reduce machine learning training times, explains how to make use of multiple GPUs with Data Parallelism vs Model Parallelism, and explores Synchronous vs Asynchronous Data Parallelism.
Mesh TensorFlow → goo.gle/3sFPrHw
Distributed Training with Keras tutorial → goo.gle/3FE6QEa
GCP Reduction Server Blog → goo.gle/3EEznYB
Multi Worker Mirrored Strategy tutorial → goo.gle/3JkQT7Y
Parameter Server Strategy tutorial → goo.gle/2Zz3UrW
Distributed training on GCP Demo → goo.gle/3pABNDE
Chapters:
0:00 - Introduction
00:17 - Agenda
00:37 - Why distributed training?
1:49 - Data Parallelism vs Model Parallelism
6:05 - Synchronous Data Parallelism
18:20 - Asynchronous Data Parallelism
23:41 Thank you for watching
Watch more ML Tech Talks → goo.gle/ml-tech-talks
Subscribe to TensorFlow → goo.gle/TensorFlow
#TensorFlow #MachineLearning #ML
product: TensorFlow - General;

[@TensorFlow]

Learn more: Mesh TensorFlow → goo.gle/3sFPrHw Distributed Training with Keras tutorial → goo.gle/3FE6QEa GCP Reduction Server Blog → goo.gle/3EEznYB Multi Worker Mirrored Strategy tutorial → goo.gle/3JkQT7Y Parameter Server Strategy tutorial → goo.gle/2Zz3UrW Distributed training on GCP Demo → goo.gle/3pABNDE

[@lighttheoryllc4337]

Assalamualaikum She is a Mushrik wearing Graven images on necklace and worshipping man made false idols. She is Breaking the 1st Commandment in Deuteronomy Chapter 5 verses 7-9

[@user-wr4yl7tx3w]

this is really well explained. more on this series please. thanks

[@giovannimurru]

Wow Ring All-Reduce is just... beautiful. 😍

[@kkmm4179]

awesome video， crystal clear with the content design and easy to understand

[@hannibalbra1216]

Hello is there any docs about federated learning with differentiel privacy, thank you

[@hasithkashyapa7645]

Straightforward and awesome

[@extrememike]

Good an easy to understand explanations!

[@chaaboudd2272]

Well done. I hop to see more videos.

[@lukasvandewiel860]

Thank you very much for the insightful presentation.

[@sudhanshuhate5633]

Nicely explained!

[@ahmadnoroozi8101]

GPU must be same, what happens if i use different GPU???

[@Ajaytshaju]

What if i dont have gpus as you said in the video, i have 32 systems with i5 CPU..can i run this mirrored strategy on multiple CPUs?

[@amanvishnoi2721]

Awsm and easy to understand.

[@tricialobo9233]

nicely done!

[@user-wr4yl7tx3w]

Great content. Look forward to more.

[@TensorFlow]

Thank you! Want to watch more? Check out this playlist, ML Tech Talks → goo.gle/ml-tech-talks

[@NoDoubt747]

wow, great job!

[@shivangitomar5557]

Amazing!!

[@VanWarren]

At time code 15:23 in the Ring Reduce algorithm the subscripts for the c vector are incorrect.

[@giovannimurru]

there is a bomb hidden in the algorithm LOL

[@advanceprogramming225]

Thank you

[@abdourahmanebalde9153]

Thank

[@TheHandbook2802]

thanks

[@liuauto]

That tf.distribute.experimental worries me.. not sure when the api will be deprecated.

[@yongqiwurosen9888]

This is awesome.