How to Generate Video - Intro to Deep Learning #15

Generative Adversarial Networks. It's time. We're going to use a Deep Convolutional GAN to generate images of the alien language from the movie arrival that we can then stitch together to animate into video. I'll go over the architecture of a GAN and then we'll implement one ourselves!
Code for this video (coding challenge included):
github.com/llSourcell/how_to_...
Nemanja's winning code:
github.com/Nemzy/video_generator
Niyas' Runner up code:
github.com/niazangels/vae-pok...
and his blog post:
hackernoon.com/how-to-autoenc...
Please Subscribe! And like. and comment. That's what keeps me going.
More Learning Resources:
blog.aylien.com/introduction-g...
blog.openai.com/generative-mo...
cs.stanford.edu/people/karpath...
channel9.msdn.com/Events/Neur...
wiseodd.github.io/techblog/201...
www.slideshare.net/ThomasDaSi...
blog.evjang.com/2016/06/genera...
/ generative-adversarial...
Join us in the Wizards Slack channel:
wizards.herokuapp.com/
And please support me on Patreon:
www.patreon.com/user?u=3191693
Follow me:
Twitter: / sirajraval
Facebook: / sirajology Instagram: / sirajraval Instagram: / sirajraval
Signup for my newsletter for exciting updates in the field of AI:
goo.gl/FZzJ5w
Hit the Join button above to sign up to become a member of my channel for access to exclusive content! Join my AI community: chatgptschool.io/ Sign up for my AI Sports betting Bot, WagerGPT! (500 spots available):
www.wagergpt.co

[@randhir1224]

I love how he always has corresponding code examples with the theoretical concepts!! :D PLEASE NEVER STOP MAKING VIDEOS :)

[@SirajRaval]

thanks Randhir! I won't :)

[@geet1993]

You make it seem so easy and clear to understand Siraj! Just gave a presentation on GANs in class last week and i adapted to your way of explaining things! Worked Wonders! Thank you so much! :)

[@SirajRaval]

great to hear thanks Geet!

[@herougo]

Nice video Siraj! I particularly liked your analogy with the magician.

[@SirajRaval]

thanks herougo!

[@Mramongus148]

Wow, the way u explains its out of the world

[@someone5781]

Love your videos Siraj, keep going! :)

[@SirajRaval]

thanks Sujeeth!

[@OzzieCoto]

Siraj, this is like drinking from a fire hydrant. :) How wonderful! Thanks for sharing your love of your craft. You are the Badass of EVERY week. And WE bow to you. :) Have a great week ahead. Peace.

[@SirajRaval]

thanks Ozzie!

[@damonki9508]

You are THE teacher!

[@SirajRaval]

thanks DaMonki!

[@deepfakestudio7776]

Wooow very interesting !!! Good job dude !

[@SirajRaval]

thanks Petre!

[@lflee]

Convoluting the hard-G-soft-G pronunciation discussion to a whole new level. Summoning Mr. "G"oodfellow to clarify.

[@SirajRaval]

lol nice

[@larryteslaspacexboringlawr739]

thank you for Deep Convolutional GAN video

[@larryteslaspacexboringlawr739]

i forgot to ask, do you have any interest in an audio only 'interesting AI news this month'? where you can just record raw audio (no editing) of you giving first impressions of interesting news headlines or research papers (banana spoon is don hertzfeldt 2000 my spoon is too big)

[@olfmombach260]

I didn't know that you already have had a video about this :D

[@SirajRaval]

moving fast!

[@dixingxu]

Wow, nice explanation. Cooooool!

[@SirajRaval]

thanks Dex!

[@user-ew2nb3rv8x]

特别棒，讲的超好

[@bonesfelipe93]

Very cool as always :)

[@SirajRaval]

thanks Andres!

[@UsmanAhmed-sq9bl]

Top Quality Siraj ! Your presentation is awesome Please let us know about upcoming video at the ending part.

[@SirajRaval]

thanks Usman!

[@davidsalleyezonme1283]

another great video.

[@SirajRaval]

thanks!

[@MichaelRicksAherne]

Jesus this is amazing.

[@SirajRaval]

thanks Michael!

[@leajiaure]

Please do an update/livestream on this topic!

[@iamgaurav_official]

siraj bro u doing very well a great thanks. mja aa gya

[@SirajRaval]

thanks Gaurav!

[@dovozamamba6704]

Dude, you are just the best

[@SirajRaval]

thanks Dovoza!

[@ryanbrown7893]

This is massive!

[@SirajRaval]

thanks Ryan!

[@TheAizaz420]

Siraj you are awesome. I had hard time understanding this. But not anymore

[@SirajRaval]

great to hear thanks Aizaz!

[@MaxLohMusic]

Wait wut there's no actual video generation here. Just generated images and then a remark about how it could technically generate video. But generating a lot of images isn't the same thing as generating a video. The video's images need to be correlated with each other in a video-like way; we can't just stitch together a ton of GAN-generated images and expect it to look like a video right?

[@ThePrime125]

actually yes, because slight changes of the input vector only change slightly the output in a continuous way, so it will look like a video, a video that makes no sense tho

[@visheshmangla2650]

club it with a rnn to get video.

[@tamgaming9861]

Awesome made video, even if i understood only 10-20% of the content because its so deep :-)

[@SirajRaval]

thanks! more intro material coming soon

[@viddeliten]

Wow, right into the code without a word of what language it is or where to go if you are new to the channel and want to start at some suitable tutorial for getting into it. Cool ideas, I have no idea how to get into this 😎

[@remenyiczn]

Everyone know deep learning is done in HTML. Whats wrong with you?

[@vishwampandya8072]

Could you in more detail explain the optimization process of GANS?

[@waxwingvain]

Hey Siraj! What are some machine learning/deep learning papers you would recommend?

[@swatitiwari18051992]

@Siraj , you are awesome :D

[@SirajRaval]

thx Swati!

[@RaisesCom]

This guy's going hard

[@SirajRaval]

thanks Natu and good to see u

[@stephens1306]

Hey raj, Will you do any tutorial video for Syntaxnet in tensorflow?

[@ranjeethmahankali3066]

Hi Siraj, A question about history: what was PDP (Parallel distributed computing) and when did people start using Matrix multiplications for Neural Networks ?

[@bitp]

Could you do a video about handwriting generation ?

[@christ2581]

* Inputs Pokemon * Generator : Here's a keychain Discriminator : Yup, that's a Pokemon

[@G3lectronX]

@Siraj_Raval Hey i`ve just tried to run your code, but i get an error, when i setup the discriminator_model at the point model.add(Dense(256)). The error says that the size gets less than 0. Any any ideas what i have to change?

[@heathervica1108]

Hello guys, do you know if is possible using: • Variational Autoencoders Neural Network (VAE) or • Generative adversarial networks (GANs) For structured data? I have seen some examples and it could be used but just for unstructured data such as images, audio, etc. Maybe do you have any example with structured data? Thanks a lot

[@harikmr2795]

Will you do video about Evolution Algorithms

[@MrHuno92]

Harikumar thecodingtrain currently makes a series on it

[@Fledron]

oh i love the coding train

[@EctoMorpheus]

Please do! I think it's an amazing concept, but when I tried to make something myself, it obviously turned out to be shit...

[@MrJorgeceja123]

Here is the repo from OpenAI: github.com/openai/evolution-strategies-starter

[@diegoantoniorosariopalomin4977]

evoltion algorithms combined with reinforcement learning would get extremly close to how animals brains work

[@tonyassi]

To get the code to work I had to install these versions of keras and tensorflow pip install keras==2.2.4 pip install tensorflow==1.13.1

[@WilsonMar1]

[4:16] Coding starts - using Keras

[@allurbase]

What i usually find hardest is knowing which shape np arrays should have, when to reshape and how to debug it

[@SirajRaval]

good point, vid on that soon

[@siddharthagrawal8300]

which IDE and compiler does siraj use for python coding?

[@ananthanir]

Will you do video about Collaborative AI!

[@juubes5557]

How do I resize different images for my input?

[@SirajRaval]

docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.misc.imresize.html

[@vineetkothari2839]

nice siraj

[@SirajRaval]

thanks Vineet!

[@MrJorgeceja123]

Why use tanh and sigmoid as activation functions and not relu or elu?

[@szymonsteczek2180]

From what I know derivative of RELu function only has 2 values: 0 and 1. Derivative of ELU is similar to RELu but is different for x < 0. Tanh and sigmoid have derivatives representation as continuous function. It is more useful to have derivative as a function of input data than a fixed number. Here you have comparison: en.wikipedia.org/wiki/Activation_function

[@Thomas-zx4uk]

"Use ReLU activation in generator for all layers except for the output, which uses Tanh. Use LeakyReLU activation in the discriminator for all layers." I followed its structure and it works well src: arxiv.org/abs/1511.06434

[@MrJorgeceja123]

Thanks! I also saw that and forgot to mention LeakyReLU. Here is the link for Improved GANs from Open AI: github.com/openai/improved-gan

[@SirajRaval]

what barras said also vid on that coming soon

[@muaazkasker5075]

That dig at Apple 😂👌

[@Barnardrab]

Your content is amazing! Are you sponsored by a corporate entity, or are you doing this series on your own independently?

[@SirajRaval]

thanks Barnard! I am independent. I collaborate with Udacity but the content is my own, no help on that

[@MrC0MPUT3R]

I'm glad to see you're almost at 100k. More people need to stop watching prank channels and learn something 👍

[@SirajRaval]

so true thx

[@MrC0MPUT3R]

Happy 100k! (technically 103k while writing this)

[@16dedikodu34]

How can we count an item type in an image? How many balls, people, cars are there? VGG-16 only gives labels (afaik)

[@carl_the_]

But what if... we trained it on a dataset of SrPelo videos?

[@yingxuezhang3825]

finally😬😬😬

[@SirajRaval]

:)

[@mrdbourke]

I lost it at 0:45 😂

[@geogeo3644]

great Pokémon rap

[@SirajRaval]

thanks geo!

[@aamir122a]

Really Nice. However please don't rush through them, make them longer if you have to get the point across.

[@erionmema1]

what kind of compuders do you need to run these networks?? does my pc have a chance of doing something interesting ? 8gb ram / i7 2600k cpu / gtx 960, Im a designer by the way not a full on programmer and Im just getting interested in AI

[@SirajRaval]

yea for sure. also google cloud is another good option. vid on cloud computing coming up soon

[@kingburrito7773]

thats fucking awesome love u siraj

[@SirajRaval]

love u kingburrito thanks!

[@luck3949]

Why generator can't just "generate" an exact copy of a training image?

[@panagiotispetridis7961]

What's the link for the Ian Goodfellow Interview?

[@SirajRaval]

kzbin.info/www/bejne/boa0q32XlpV7hrc

[@panagiotispetridis7961]

Thank you!

[@celsolopes2639]

Hi, Siraj. Do you have any video about INFORMATION RETRIEVAL using Deep Learning? If you have, can you pass the link?

[@FinnT730]

What website do you use to make this?

[@SirajRaval]

final cut pro

[@WhiteDragon103]

Link to source at 1:57?

[@Fledron]

The one dislike is by Apple i guess?

[@SirajRaval]

always

[@NickKartha]

Notice that the number of dislikes on this video is equal to the number of likes on @Fledron's comment

[@MaxLohMusic]

​@@SirajRaval It seems like the video is titled how to generate videos but you really only taught how to generate images. I think when people talk about video generation they're not thinking of a slide show of images that have nothing to do with each other, but an actual video with care about how the frames make sense across time. What are your thoughts on actual video generation with GANs?

[@ivartheboneless6158]

U r just cool

[@Skythedragon]

He's a good fellow :) Also, If I would put G in a robot, and D in a camera that looks at the robot, I could train it based on videos of humans doing things, eventually making a robot that acts extremely human? (In theory, I won't build skynet)

[@SirajRaval]

if the robot had the necessary degrees of freedom as we do, then yes absolutely! great project idea

[@Skythedragon]

Nice to know, I guess my new 3d printer comes in handy then! Also, is there a neural network training model that works based on the activity of neurons? (similar to how hoomans learn)

[@AssasinMilo]

Skythedragon I believe they're called artificial neural nets, check Wikipedia, although these only mimic axons and action potential of neurons from what I understand and real biological systems are much more intricate as there are many other features in the nervous system that contribute and interact with the neurons ability to fire action potentials then just dendritic input alone, I could elaborate more if you want I'm a microbiologists and I got an A in neuro.

[@RahulBonnerjee]

Do you use ML to generate all of the amazing memes you use? Could you train a model to make the best topical memes??

[@SirajRaval]

nope hand picked but yes you could for sure

[@RahulBonnerjee]

Yesss together we can make millions... of people follow us on instagram!

[@anoncomenter2469]

Ouch is this PHD level? or what level and how many degree years experience do you have to do this stuff.

[@quebono100]

One answer never lets me go. Why we never heard about Chinese, Russian or east-Europe scientist. Are they bad? I must truly admit that the most interesting books I found on the US-Market. What's your opinion about that Siraj. Thank you.

[@SirajRaval]

The Chinese are getting really good, right now US dominates but time will tell! I hope to see more global breakthroughs

[@L0Ls0ul]

8:55 The real question is: Is the shadow real or animated by an AI. Kappa.

[@SirajRaval]

real questions ftw

[@diegoantoniorosariopalomin4977]

hey siraj , i have been reading about the state of the state of the art in fps game bots , and have some questions

[@diegoantoniorosariopalomin4977]

also , that model you talked about probablty uses a recurrent layer , to remeber past frames and stay temporally coherent , videos are more than a collection of images , should have mentioned that considering the coding challenge

[@diegoantoniorosariopalomin4977]

btw i understand what recurrent nets are from game bot research papers and pots , i think ( i want to talk about it ) bots use a lstm to remember what they have done to avoid doing the opposite next , and the ones in 3d enviroments also use it to remebers enemies offscreen and such

[@SirajRaval]

cool what question?

[@diegoantoniorosariopalomin4977]

Do cnns computation requierement scale linearly with resolution ?

[@diegoantoniorosariopalomin4977]

Did a3c made q learning obsolete ?

[@davidmata3104]

Normalizing well could get better performance and converge quickly. Most of time we will subtract mean value to make input mean to be zero to prevent weights change same directions so that converge slowly ReLU nonlinear acitivation worked better and performed state-of-art results in deep learning and MLP. Why are you using tanh instead of using relu? Tanh introduce the vanishing gradient problem, isn't it

[@SaikatBasak]

I have implemented semantic image completion using DCGAN. Here's the source: github.com/saikatbsk/ImageCompletion-DCGAN

[@nikhilchhabra]

nice work. comment about image generation : it seems that d generated images have existing faces like shahid , irfan, arjun rampal. (what if it is just picking faces from training sample ?) Correct me if am wrong.

[@SaikatBasak]

Nikhil Chhabra the dataset i used contains images of indian film stars. the generator is recreating those images in an unsupervised/reinforced way, from a random uniform distribution. and it can actually generate realistic images although there might be some perceptual difference.

[@SirajRaval]

nice work Saikat

[@chrisoffersen]

How do I make money learning this stuff??! lol ...seriously (because my job is getting in the way of doing this full time)

[@SirajRaval]

think of a problem you'd like to solve. 99% of the time ML can be applied to solve it. profit

[@chrisoffersen]

Bam, nailed it. ....soooo many ideas...so little time. ...I need a network of people and maybe a new city to live in. Thanks Siraj! You're doing great things and making sparks fly. We'll see what those sparks ignite.

[@MaxLohMusic]

He didn't explain how to generate videos. He explained how to generate images and then just said to stitch those images to form a video. That's obviously not what we're looking for because it's just a slideshow of separate images instead of a video as a whole with transitions between frames that are realistic across time.

[@ThomasOrlita]

Why am I watching this when I don't understand it at all.

[@luck3949]

To understand it?

[@ThomasOrlita]

I understand the basic theory (kind of...), but when it comes to writing ML code it's quite difficult...

[@haripatel2586]

Hi Siraj. I study maths and physics and am about to graduate. I would like to master AI in my life. How would one start off from your perspective? I am 21 and have a fear that it is too late.

[@szymonsteczek2180]

Question to everyone: Does anyone know any blogs or tutorials for applying algorithms from academic papers (or similar topic)? I almost forgot. Thanks Siraj for video!

[@szymonsteczek2180]

Thanks a lot!

[@DannZylman]

Who wants to build an ai with me

[@SirajRaval]

me

[@DannZylman]

Thanks you already helped me a lot with your video's

[@AKASH-zt8oi]

Sir, could you help me with my code. I am trying to do the above video generation using conv3D but couldn't get the dimensions right for my generator

[@Eddie-rf4tp]

Siraj, you have a great community. I will give $1 to cut the musical segments from these videos. :D

[@SirajRaval]

thanks Eddie! Ya this one couldve been better ha

[@Melvin420x12]

But why would you let your machine learn something from scratch? We don't let our children learn everything from scratch neither do we? Maybe an interesting idea to find a way to explain an AI what this alien language is and then show it the training data? Maybe it'll learn faster, or maybe it's a flop. Machine learning is far beyond my knowledge..