Differentiable Neural Computer (LIVE)

[@Collinoeight]

This is easily the best thing I've seen all month. Probably all year.

[@martinpetersson4350]

I feel the same!

[@SirajRaval]

awesome to hear

[@o0xemas0o]

Man, I love how you explore both the theoretical high level understanding as well as the nitty gritty. There's a lot of technical jargon that can be confusing, and I have found that people who don't really understand AI will focus entirely on the minutia.

[@SwanandKulkarni2194]

Loved it. Code walk through was so precise, & easy language explanation for everyone's understanding. Just great. Extremely excited about the upcoming stuff, eagerly waiting.

[@SirajRaval]

awesome

[@WildAnimalChannel]

Lots of people tried it but they never got it to work before! Deep mind are very clever people!

[@larryteslaspacexboringlawr739]

thank you for Differentiable Neural Computer video, helping me learn a very complicated topic

[@JM168M]

Siraj you should write books base on all your teaching here. Make it easy to understand just like your videos!. Love you Man!!

[@chrisfaraday3924]

Videos and courses > Books

[@MrNtuer]

Easily the best rap among what I have watched this year!

[@SirajRaval]

awesome

[@simdimdim]

didn't sound like rap, more like rock and roll :D

[@nitink4245]

siraj , you are the best

[@Arjun-jt7yb]

one of my favorite videos that I like most.

[@CarterColeisInfamous]

your totally right we need hardware and the software should program the hardware

[@fayezbayzidify]

Yo Siraj thanks to your awesome ways of teaching I made one before google but instead I used a seed to encode the memory

[@kazinazmulhaqueshezan4219]

Good job buddy!!! Good job!!!!! You are making my path easy....Thanks a lot, man!! I was just started to read the paper.

[@SirajRaval]

thx

[@_XoR_]

It will be so cool to use this into some sort of low level hybrid fpga system so you could have it reprogram itself with each new problem. But I bet google is already working on custom hardware.

[@coleflicek9053]

Question. This seems like a very complicated way to build nested neural nets. Where the first NN is simply the input to the memory matrix and the second NN is another input. Am I understanding that correctly or am I over simplifying things?

[@luisleal4169]

I've read the paper, seen this video several times, and still cant figure out where the "train on 2 tasks" part that you mentioned comes in, can you provide an example with some meaningful data instead of random data?(maybe some graph problem like the underground or family tree mentioned) and using the "2 separate data types" part you mentioned, sounds amazing but im having hard time figuring out where to fit that.

[@lillicite-centralelille5572]

Yes I don't understand either. I hope we will get a response.

[@yena-coco]

same idea....

[@demochannel6146]

How does it understand the order of the problems ? Like how can it understand to process text first, then graph problem ? Why it doesn't go the graph NN first ?

[@poojanpatel2437]

i just though abouth this thing 2 days ago and thinking if it is possible or not training a neural network to do multiple thing .. And i am so excited this can be happen.. cant wait to know more about this topic

[@raju5081]

will there be any live sessions again on ML and Deep Learning?

[@aryopradiptagema6677]

is there any essential difference between your implementation and deepmind released source code?

[@t2kien]

You defined sefl.num_heads=R as the number of read+write heads. But according to the DNC paper, R is the number of read heads only. What is the difference in your code?

[@yingglay]

I have two questions: 1) Could you please explain the prediction matrix in the last part of your example output? I only understand input and output random matrix. 2) You mentioned it is possible to define two problems using external memory, voice recognition translated to question, and then using question to find the underground optimal route. But I do not know how to do it, could you provide an easiest example in github repository with image recognition translated to input matrix, and then input matrix -> output matrix. I guess two predictions are needed, do not know if I understand dnc correctly, thank you very much.

[@secondsandthings]

Can this be used for 1-shot learning? Does it perform worse?

[@McMurchie]

Hey Siraj, this is awesome as always having you at the forefront of the cutting edge and relaying it to us. Do you think we could further advance this by connecting two Differentiable Neural Computers together - so we could have say a graph problem, an image recognition problem, text based questioning problem and say a translation problem to have it use common learnings to apply to all 4 problems?

[@joshpceeg]

How can I use dnc on my data? I see many github examples some I get to run on python3 ubuntu 17.10 tensorflow and sonnet and my gpu but I dont know how to format a spread sheet of data or a folder with data or python sliding window for time series data?

[@henrywong7286]

This is one cool video! Thank you so much for the effort! But can someone tell me where comes the generalization characteristics of this new architecture?

[@rufiromang]

hi Siraj. thx for the awesome video. do you actually have a website where you post this code?

[@SirajRaval]

see vid desc yes

[@rufiromang]

thanks a lot Sir!

[@mehulkumarnirala8769]

Can you please make a video tutorial on distributed training using tensorflow?

[@jacobusstrydom7017]

Hay man your channel is awesome!!!!!!

[@stefanogrillo6040]

when you start a channel as a comp scientists, but terminates as freddie mercury imitator 🤣

[@hoangan07]

18:42 We are going to focus on theory, and we are going to focus on your hair. :))

[@randywelt8210]

saraj beyond stanford lectures and yann lecun presentations. 😋

[@SirajRaval]

i know right

[@atifadib]

Link to the paper please!

[@nitink4245]

siraj please let us know if we can use , one shot learning or mann for fraud or loan defaulter prediction ??? . if so please make a comprehensive video ?

[@simdimdim]

24:03 I got seasick..

[@AviPars]

can you do some linear algebra courses or suggest some for beginners

[@fgfanta]

www.khanacademy.org/math/linear-algebra

[@avatar098]

KhanAcademy

[@chicken6180]

homework done early looks like im bingwatching khanacadamy bois

[@chonglv9766]

Cool rap. Maybe it's better to implement the bAbI task to replace the random data.

[@MarioSalvini]

Weights and ExtMemory ... 'to Have' and 'to Be' ;-)

[@nadeemn4100]

Dude...y aren't u posting new videos on deep learning

[@FilipeSilva1]

THIS! oh yes

[@arminherbsthofer2815]

@Siraj The predictions of your network are complete nonsense. Use loss = tf.reduce_mean(tf.pow(output - dnc.o_data, 2)) to get something sensible. And why is your training data initialization so overcomplicated? final_i_data = np.random.randint(0,2,size=(2*seq_len, input_size)) and final_o_data = np.random.randint(0,2,size=(2*seq_len, input_size)) is enough.

[@yeahokaysureyoubet]

But can this DNC learn to love Bernie?

[@joespider8647]

god damn u youtube so many commercials

[@spayseghost]

differentiable sounds like bad enlgish lol

[@АнтонДостоевский-ж2ш]

Why Mac OS so nice, and windows so ugly ?

[@IroshanVithanage]

Windows looks great :(

[@NathanK97]

because mac users are only interested in shiny things and dont look at what the thing capable of

[@rrr00bb1]

if you are doing development, the unix underpinnings of OSX are a really big deal; as you will be deploying to linux machines (aws, google cloud,etc). whenever we get a team mate that uses windows, it always ends up turning the build into a mess; unless we are actually building binaries that will be distributed on windows machines (in which case it makes sense).

[@stefanogrillo6040]

when you start a channel as a comp scientists, but terminates as freddie mercury imitator 🤣