How to Learn from Little Data - Intro to Deep Learning #17

[@khanfor]

You make ML so accessible and convenient. Keep it up Siraj! You're an awesome educator!

[@SirajRaval]

thanks Abdullah!

[@xPROxSNIPExMW2xPOWER]

I learn from little data all the time, it's called college education

[@xPROxSNIPExMW2xPOWER]

Lemmas are the bane of my everything lol kmn

[@swapanjain892]

That's not learning,thats called "How do you waste your time"?

[@userou-ig1ze]

ProSurviver you're too optimistic

[@marvlousdasta2566]

and most of this little data are noise

[@Aer0xander]

Get little data for going in debt

[@PsRafael84]

I have to say: im not a programmer, im a doctor with programming as some short of a hobby. But your videos actually make me rethink about my profession! The way you teach, the quality of your videos! Man! Thank you for sharing your knowledge. Hope for more videos. Success!!

[@SirajRaval]

thats aweosme thanks!

[@SchultzC]

Someone give this man a medal. These videos have been amazing.

[@markph0204]

Mix of learning and humor nicely done.

[@olfmombach260]

I have three questions: 1. So there is really nothing more than the RNN (controller) that uses its memory database to achieve the best results? 2. Is this implementable in keras? 3. Should I be more focused on keras (for efficiency) or tensorflow (for flexibility or accessibility)? Are there things that can be done only in tensorflow?

[@pdsnk1]

Thanks a lot for your videos! They are so many! I even don't have enough time to watch them all!

[@SirajRaval]

thanks Albert

[@avatar098]

Dude, Siraj, you helped me with one of my kaggle projects that contains little data (like 8-10 examples/label). This is my first time being exposed to concepts like neural turing machines :) You have earned a subscriber

[@misterj9817]

man, i love this guy ! I think i am NOT the only one who likes the video first then watches it.

[@SirajRaval]

love u

[@chroniclesofave2137]

Great video Siraj... You're like my role model in my Deep Learning journey and ML in general. I've learnt so much within such a short time. Keep up the great work and hope you reach far. Watched all your videos by the way.

[@bernardogalvao4448]

I missed watching your videos :') Recently I decided to finally read the paper (which I learned about because of your video back then) after a year long of putting it off... thankfully Siraj comes to the rescue breaking it down in more digestible pieces :) keep up the good work

[@secondsandthings]

Woo! Was waiting for this for so long. Great video Siraj

[@SirajRaval]

thanks!

[@bobcrunch]

And the crowd goes wild! Siraj spikes the ball and the crowd pours out on the field!

[@SirajRaval]

thx Bob!

[@michaelyadchuk4989]

Congrats on the completion of the course. I'm looking forward to taking your Udacity Nanodegree!

[@SirajRaval]

awesome thx

[@barbolo]

Great video, Siraj. I'd like to suggest you to explain Generative ladder networks (REGEL) for problems with small labeled datasets available, but also with big unlabeled datasets.

[@florentinalexandruiftimie8214]

Siraj, you are freaking awesome!!!

[@SirajRaval]

thx florentin!

[@RoxanaNoe]

Great great video. You totally rock Siraj!!!

[@vaer-k]

Can you please share links in the description to the journal articles you highlight in your videos?

[@franklinbradfield9131]

Very powerful concepts here. It's a lot to package into a 9 minute video, so I've been going over the arXiv paper to get a better grasp of the fundamentals. If you have a moment, can you comment on the difference(s) between one-shot learning and transfer learning?

[@troy3393]

SIRAJ! Not sure if you remember me commenting a few months back about starting my Data Science degree and a bunch of Math being involved but I need some more guidance. We have a project where we're exploring simple regression and multivariable regression models on a dataset of our choosing. I want to try take it a step further. I have a bunch of individual player data for a sports league (seasons 2012 - 2016), and I have a source of the data so I could probably grab the 2017 results with a little wget magic. ANYWAY, my goal is to predict a future win or loss when one team is playing another (sounds like a 'simple' classification problem - win or loss) taking as many individual player statistics into account as possible. Is there any easy ways I can just throw the data into a pile, tell the computer a WIN is good and a LOSS is bad (a DRAW is okay); and then get it to spit back out some correlations that it found with a win or a loss when specific teams are up against one another? I need a little bit of a starting point - what should I be looking at? Would love to shock my class and lecturer with a semi-intelligent model!

[@calebkirschbaum8158]

Could you use this to train a network to make a network that is supposed to make networks using the original network as the data? Basically, Make 'A' be able to make 'B' that can make 'C', by using 'A' as the data for 'A'. Then using all subsequent programs use that for more data to become better? Or is that way too complicated for one shot learning?

[@I77AGIC]

such a unique channel. and always great info. thanks!

[@SirajRaval]

thanks!

[@TheOjayyy]

loved the course, thanks, keep up the good work man!

[@SirajRaval]

thx ojay will do

[@LaSalaProductions]

if I want to create a Q&A system that answers from a book chapter, should a use seq2seq or DMN?

[@waxwingvain]

Finally! One shot!

[@SirajRaval]

finally

[@HmmContemplates]

I love it, never change Siraj. Keep on learning though

[@SirajRaval]

thx i wont and will

[@manalmohammed27]

Siraj, is it feasible to implement it with text classification ?

[@xzhao5824]

Hi,Siraj.where can I find the winner code for this week's competition?

[@nitink4245]

siraj please let us know if we can use , one shot learning or mann for fraud or loan defaulter prediction ??? . if so please make a comprehensive video ?

[@sryan2640]

Great channel dude, gonna fucking tear through all your ML videos when my exams are over

[@BrunoJuncklaus]

Note that you also need pytest, matplotlib, Pillow and scipy to run the code. And also make sure to add a __init__.py in both MANN/ and MANN/Utils/ folder. And also make sure you create a file ~/.matplotlib/matplotlibrc there and add the following code (if you're using virtualenv): backend: TkAgg And even though when you're trying to sample the self.character_folders it raises an error, because the variable is empty.

[@ionmosnoi]

in order to make it work for tensorflow v1, is it enough to change batch_matmul to matmul, mul to multiply, concat_v2 to concat and add labels, logits keywords for softmax_cross_entropy_with_logits?

[@SirajRaval]

thx bruno yes this code was tough AF

[@danieldeleon5804]

Siraj, I would like to hear a bit more about the life cycle of a deep learning paper. Can you make a video or point me in the direction where I can learn more?

[@SirajRaval]

ill make a vid on this

[@vineetkothari2839]

As you show it is used for image classification can it be used with CNN and GAN if yes how ?

[@SirajRaval]

hasn't been tried, good experiment!

[@ebtesamh9624]

Can we use this for translation

[@macot79]

Doesn't the central limit theorem kinda indicate that using a Gaussian distribution can often times be a good choice of distribution when trying to model a lot of phenomenon?

[@SirajRaval]

indeed just saying we r still figuring out things

[@datasciencewithr1039]

Thanks for the video! Get well soon Siraj!

[@SirajRaval]

thx. get well? im not sick tho lol

[@chrisoffersen]

Can't wait until I understand this!

[@chrisoffersen]

For context: I'm only on Submission 2 in Udacity's Deep Learning ND

[@SirajRaval]

so close

[@charansai9421]

nice video siraj and?? can u suggest a good book for learning algorithms??

[@UsmanAhmed-sq9bl]

Siraj: great tutorial as always. After this course, what is inline ? Please give detail of your upcoming videos.

[@SirajRaval]

thx figuring it out

[@elfamosodpereira]

Would this work for a chatbot project?

[@cidroldan6732]

Hello, great vid! also where can i see the code for this lesson's challenge winner?

[@SirajRaval]

Thanks! none this video

[@allanng78]

Hi Sir Siraj, I have downloaded and run the code in github. It seems to take forever. Is it suppose to be like that? Please help..... I am running this in a vn in an i7 laptop.

[@shivambhardwaj5599]

Hey Siraj, I just want to know how I can learn Machine learning for autonomous cars from scratch and what language should I choose. Thanks Your Videos are really intersting.

[@elisayao668]

Hi Siraj, this tis a great one! Could you do one in information leakage? e.g. processing testing dataset using training set statistics.

[@bliss766]

Please do a video on zero-shot-learning.

[@kingpopaul]

Has anyone tried the Hierarchical Temporal Memory (HTM) algorithms from Numenta, I'm curious about their tech but I don't know if they work better than the more common neural networks solutions.

[@SirajRaval]

numenta is kind of a joke since they hardly publish but make big claims

[@kingpopaul]

Yeah they publish a little but I'm keeping an eye on them, their github is active and they are gradually adding stuff to their nupic libraries. Maybe once they have implemented enough algorythms it'll be interesting to compare and see if their claim are true. Though for now they mostly seem to focus on anomaly detection for their products, which is pretty lame in my opinion. But if their 'novel' approach is any good, it could turn out to be interesting. Thanks for the reply kind sir!

[@theosalmon]

The claim is that they're reverse engineering the neo-cortex. If that becomes successful, it will be an invaluable contribution. Practical applications are minimal so far, but the mechanisms they propose are very different and interesting.

[@Renardbardhi1]

Siraj nice work. I would like to ask for learning from a small data set. I am using Corell5k data set that contains 50 classes each class contains 100 images and its small data for CNN to achieve a high score of accuracy. How can I increase accuracy? Thank you very much

[@ebtesamh9624]

Hi, can i apply this code to text data

[@Lindomcgud]

How big is you dataset?

[@gabrielsb15]

it doesnt work for python 3.5 right?

[@MaduSudananB_N]

So who taught you machine learning in the first place? Are you self learnt ? P.S : Thanks for all these videos, they are amazing.

[@artr0x93]

Great video as always! Is there a discord channel for Siraj-related things? If not I would really like to see one created, so that ppl can go there and discuss the challenges. Discord is really good at bringing communities together in my experience.

[@SirajRaval]

we use slack see vid desc thx

[@Skythedragon]

I sadly doesn't take one shot for me to understand this, But great video, I've been wanting to train models on tiny datasets!

[@SirajRaval]

thanks!

[@Skythedragon]

+Siraj Raval also, what's the difference between a MANN and DNC?

[@bf2825]

Hi, Siraj. I want to ask about why people in ML always discourage people from learning Deep Learning?

[@ngrislain]

Yes, the gaussian is mathematically convenient, but it is also special in the sense it maximises Entropy for a given (mean, std-err) pair, among so many special properties...

[@SirajRaval]

good points

[@siddharthyadav5682]

Most of the times, I watch your videos just for recreational purposes.

[@aryopradiptagema6677]

This is what i was waiting for!!! BTW, mathematically speaking, any kind of memory augmented neural networks actually can be used even to solve a problem with a moderate amount of data (not too big, not too small), right? Any thought about the performance comparison between other deep learning algorithms and MANN in task with a moderate amount of data?

[@SirajRaval]

awesom thanks and yes. sounds like a good experiment

[@calebkirschbaum8158]

Can you not combine the 2 so as to make deep learning really effective at lower levels, and it just keeps getting better with more data you throw at it?

[@logicboard7746]

Listening to Siraj = time well spent

[@SirajRaval]

thx logic

[@vigneshyaadav6322]

@Siraj: please do make vedios about robotics u may have a grate. knowledge as u had been in a start up

[@alisyariatimohdsalleh1848]

The video and github code not the same. Simplified version i think.

[@larryteslaspacexboringlawr739]

thank you for deep learning video

[@diegoantoniorosariopalomin4977]

hey Siraj , i was asigned to be the representative of programing 101 in my college , any tips

[@diegoantoniorosariopalomin4977]

what to post on the google + group i created and such

[@diegoantoniorosariopalomin4977]

there is an exam on monday btw ( any resources you know ?)

[@SirajRaval]

thx diego, make a git commit everyday

[@AkashMishra23]

You should link the arXiv/ Research Papers in the Video in the Description, It'll be a lot easier to get to them...

[@DojoGiantShanNY]

Thanks Siraj, really helpful vid. you sings awesome, by the way:)

[@chinmayeejoshi2119]

Was he trying to start a the equivalent of getting rick rolled with the Barbra Streisand thing?

[@vaer-k]

Here is the link to the Neural Turing Machine deepmind article arxiv.org/abs/1410.5401

[@brofessorsbooks3352]

you are so good! god bless u!

[@SirajRaval]

thx u 2

[@JakeDownsWuzHere]

[@SirajRaval]

thanks Jake!

[@AnanthNatarajanS]

YOU ROCK.

[@andreamarkos]

brilliant!

[@alonkellner5375]

Could you make a video about ZERO DATA LEARNING too? A very interesting topic in the realm of meta-learning.

[@terryr9052]

Can you explain that more?

[@alonkellner5375]

Terry R Zero data learning is the concept of learning to do new given tasks without ever training on it (therfore the name zero data learning) using the knowledge learned from previous tasks, for example learning to classify images of digits to 0-9, and then being asked to classify to new classes like letters A-Z, and with the knowledge learned from the digits classify the new letters images without ever being trained on letters at all. The paper about zero data learning is in this link: www.cs.toronto.edu/~larocheh/publications/aaai2008_zero-data.pdf

[@terryr9052]

Ah! So a specific type of transfer learning... That is interesting - thanks for the link.

[@SirajRaval]

nice i should

[@bonesfelipe93]

Just like Oracle Hahahaha :')

[@SirajRaval]

lool

[@TidalMaker]

I died at the Pork Pig homage.

[@SirajRaval]

haha

[@yeahokaysureyoubet]

Oh MANN!

[@SirajRaval]

vietMANN

[@Pietrabentivi]

HI LEVEL CHANNEL !!!!!

[@whatthefunction9140]

Behave like an excited neuron.

[@Eddie-rf4tp]

First half was great. I don't know why, the singing kills it for me. Suggestion, sing at the end.

[@SirajRaval]

thx eddie. i gotta do me. love u

[@Eddie-rf4tp]

Haha, fair enough. Love you the same 🤓

[@walterhugolopezpinaya5641]

Who won this Coding Challenge?

[@AhmedShariffGplus]

"just like oracle" lol dude :D

[@Arty66Click]

>That Chappelle meme tho) lmao)

[@SirajRaval]

lolz

[@moazim1993]

Eww the code is in Python 2

[@God-po4rt]

I... I don't understand.

[@SirajRaval]

lmao

[@machapuis]

yaayyy

[@SirajRaval]

yay

[@synetic707x]

Programs that learn how to learn... it's getting out of hand

[@shutinghuang1398]

Love your video! I have no background in computer science or coding, but managed to train my own image classification through your tutorial! Can you do a tutorial on semantic segmentation? something like this is really fun: github.com/warmspringwinds/tensorflow_notes/blob/master/fully_convolutional_networks.ipynb

[@roshanrane3112]

Misleading title alert! This is about meta learning and one shot learning and it's definitely not an "introduction to deep learning" topic

[@htconezm4501]

just like Oracle lol

[@SirajRaval]

;)

[@terryhintz4521]

memes overflow

[@SirajRaval]

so many

[@oneroadgoodwalk]

nice video but the memes get very distracting

[@SirajRaval]

thx for feedback

[@andrewkoh6124]

I think the memes are a fresh breath of air! otherwise it would be kinda dry. thanks siraj!

[@김기완-c5r]

Who won this Coding Challenge?