How to Generate Your Own Wikipedia Articles (LIVE)

[@sunilsharma7067]

Siraj, great video. Love the useful comments on the code, absolute essentials for accessibility. The fact that the code doesn't worry about implementing an OOPs heavy flavor is an added plus. Cheers!

[@SirajRaval]

Thanks Sunil!

[@joshua_dlima]

ngl finding this channel feels like i hit a gold mine, thanks a tonne

[@dhmspector]

Hi Siraj - Great content as always; you are helping me learn a lot! As these tutorials -- and especially these live-coding sessions -- get longer and more complex I'd like to suggest that at the beginning you talk through a section oriented outline of what the video will show. And, if at all possible during the video a sidebar/overlay to show viewers where you are in terms of working thought the video as you go would be really helpful to help people keep track of where you are in a given talk and what they're going to be working on in a given section..

[@SirajRaval]

David, great feedback will do thanks!

[@pure_virtual]

That was some next level live! You are definitely building an ML entertainment empire :D

[@SirajRaval]

Thanks! It's happening

[@blipzchitz9047]

i,d really like it if you showed the working program output examples at the end of the videos, all ur vids just abruptly end like ''there you go, try it if you want to see it working..'

[@minhhieunguyen3952]

Great works. Thank you Siraj and the best support team.

[@nihshrey]

Holy Moly VR. you are creating a sort of machine learning empire over the internet. Cheers bro.

[@DutchmanDavid]

I appreciate you using VR to explain LSTM! :D

[@SirajRaval]

thanks!

[@DosellIo1]

Hi Siraj, excellent video! I am in Udacity deep learning course, it is a great fun to learn from you! cheers tibor

[@SirajRaval]

thanks!

[@derasor]

You are great Siraj. Kudos to you!

[@SirajRaval]

thanks!

[@muhammedbaldeh559]

I can't understand why the sample function have to be used? Can't the char to be output be selected from the highest prediction which is fed into the sample function?

[@MikhailChernoskutov]

Siraj, that VR thing is really awesome! If only I could use it in my lessons.

[@SirajRaval]

thanks Mikhail! I'll try to keep incorporating it

[@avananana]

I mean, ignoring the fact that you seem to assume everyone has 128 gigs of RAM, it was a prety solid way of showcasing how to use Tensorflow.

[@hnkulkarni]

The use of VR was awesome.

[@SirajRaval]

thanks!

[@ameynarkhede8264]

The new setup so awesome!

[@SirajRaval]

thanks Amey!

[@mohammednagdy6661]

At 9:54. So Siraj why can't we train the lstm word by word not character by character?

[@mikrchzichy]

@Siraj ... quick observation. while doing the live stream rather than have someone interrupt you with edits / have a them either have a txt file visible or edit your code with comments you can scan over

[@SirajRaval]

thanks for the sugggestion

[@larryteslaspacexboringlawr739]

Thank you for another Tensorflow video

[@RojinaPanta1]

Great Video.....Looking forward to one with TensorBoard

[@SirajRaval]

Ah, yes. Great point. Thanks, that is coming next week

[@johannesgh90]

Can you do Google Cloud ML and how to write TensorFlow code for it, preferably including TensorBoard and ValidationMonitor?

[@SirajRaval]

Great suggestion Johannes, yes i will absolutely do a live video on tensorboard next week

[@DeusExMach]

I love how Siraj's answer to the Venezuela question is basically "Git gud, noob!"

[@SirajRaval]

haha

[@HarshBhikadia]

Hey Siraj. nice work. TIP: use drawing tab if possible for explaining concepts. visual representations are really helpful. basically drawing LSTM and writing code for it parallely, also it would be great if you also label placeholders on the drawing with the variable names. as TF is a graph representation language... a drawing would be really helpful

[@HarshBhikadia]

I wrote this comment before watching the VR part. just ignore the comment. VR is way cooler than 2D drawings. i underestimated you :D :P

[@HarshBhikadia]

also you should have theory-only video [no coding]. eg: explaing LSTM in theory...

[@roopal_garg]

Siraj, I have a couple of questions: 1) A learning rate of "10", havent seen that being so high before. Is that because of the large batch size? Do you think a decaying learning rate would be better? Or was the "10" selected just for demo purposes. 2)I have trouble understanding the sample(prediction) function. Its purpose as you said is to select the char with the highest probability from the given prediction probability. Why did we not simply do an "argmax" across the probability distribution?

[@SirajRaval]

1) 10 is pretty high, just for demo purposes, decaying would've been better yes 2) we could've done an argmax as well

[@roopal_garg]

thanks for replying. Sounds good.

[@Mathaniac]

np.zeros(...) produces a huge array that maxes out my ram and gives a MemoryError. I tried adding dtype=uint8 but that hasn't solved my problem. Anyone have a suggestion?

[@piponwa]

Same error here

[@StitchedChin]

I added sys and ran getsizeof to see what would work with my PC. Around 6GB was about as big as I could go for it to compile, probably not the way to go, but I adjusted the for loop by dividing by 5: for i in range(0, int(len(text)/5) - len_per_section, skip): On a VM machine with Ubuntu which has much less, I had to adjust by 100.

[@kouohhashi]

I truncated text. like "text = text[:288556]". Naturally result gets ugly but still we can test entire code.

[@roopal_garg]

try this: """ the issue is that creating everything as one hot encoded for the entire dataset at once consumes a lot of memory. So lets just store the indices for the characters and only convert the matrices to one-hot form when we need to process them """ X = np.zeros((len(sections), len_per_section), dtype=int) Y = np.zeros((len(sections), len_per_section), dtype=int) """ later, before feeding your batch to the lstm convert from indices to one-hot encoded form """ input_embeddings = tf.one_hot(batch_data, depth=char_size, dtype=tf.float32, axis=-1) This solved the issue for me. Hopefully works for you guys. Let me know

[@SaurabhRathor092]

great suggestion. But can u share ur code link...

[@000FURQAN]

Siraj . really appreciated your efforts for teaching MI related stuff. I wanna ask which NN is better to predict wind , solar and electric load forecast and if possible plzz do a video on this topic.. thankxx

[@elmagnificent8550]

Hey Siraj, Thanks for your effort !! Can you be honest? I have basic programming skills ( javascript), but mainly I'm more into design aspects and need rapid prototyping for small things. Can I learn what you are teaching without having good programming skills and well apply it in my job? Thanks!

[@SirajRaval]

Hey Petras, yes. With libraries like Keras and Tensorflow, anyone can do deep learning

[@llllloolllll]

You are great, simple and powerful content.

[@SirajRaval]

Thanks Jorge!

[@chengyuhuang5966]

Hi Siraj, should we use embedding layers here? I tried out your code. I maxed out my 16g of ram in the one_hot encoding part. For some reason, my raw data seems to be 15 times bigger than yours. it seems that we can not really scale the one hot encoding part using your code.

[@bfvto]

Thanks for the video, what if we use BasicLSTMCell from TF directly, will that be better ?

[@sukumarh3646]

Two thumbs up! Can you tell me what exactly are graph and session?

[@SuriyaNarayanan987]

Hey Siraj, just curious.. which apple laptop is yours? were you training on CPU?

[@rahil1304]

Does this work with python 3.7? Or only with lower python versions? Since I'm getting this error "#concatenates (combines) vectors along a dimension axis, not multiply"

[@SamimEkram1995]

hey I wanted to know if you are having a repository in git where the live Q&A question and answers are stored ??? and if not I would very much appreciate it if this is done and I know 5 minutes is very less time to answer so if you have/make the repository then please include detailed answers too ... Thanks again

[@JithendraParuchuri]

VR for LSTM is cool :)

[@SirajRaval]

Thanks Jithendra!

[@Moonwalk3003]

Why are we taking w_ii shape as [char_size, hidden_state] and not [len(section),hidden_state]?

[@n.karamousadakis]

"We are just defining these gates that's all we are doing we are defining gates. Gates have to be defined. Bill Gates doesn't need to be defined, he is already defined by Microsoft." Siraj RaGod 22 March 2017. (46:29)

[@SirajRaval]

Lol thanks Nick

[@niramoyal5405]

Hey the Dataset link is broken - can you maybe upload it again Thank you

[@caveman36]

Shouldn't the memory cell use a tanh activation?

[@bapireddy5790]

Hey Siraj any plans to do a video object detection using tensorflow ?

[@SirajRaval]

great idea, will consider thanks

[@donnieashok7978]

How would you go about building a system where the output is an automatic summary of the concepts in the input. For e.g. if the input text is 3000 words long and contains answers to 'What is THC?' and 'How to vaporise THC?' This video shows that I can feed the script a lot of text from wikipedia and research journals, and start the output with: 'THC is..'. And the script would automatically write the rest based on the content found from the input feed. What do you say? This can solve a lot of education problems.

[@SirajRaval]

see my vid 'how to build a text summarizer'

[@MADHUTRIPURANA]

I get this error NameError: name 'reset_test_state' is not defined, though I have reset_test_state define. Any thoughts

[@saitejach9360]

Is deep learning nano degree worth it and is it from scratch or advanced

[@iProcrastinatexx]

hey, My gpu don't support tensorflow. I'm gonna start tensorflow cpu. How good is it?

[@bfvto]

Newer version of TF changed tf.concat's arguments order, so the code wont run to the end.

[@unusualtechy4566]

@Siraj Thank you for video. Can you guide me if i have own set of data (Images) for let's say Geology area of Gujarat-INDIA only, around 6000 and wants to build LENET-5 on top of that what are the steps you suggest me to go through. I have i5 CPU, ArchLinux.

[@riochuong105]

I am not able to run your demo code due to memory error even on AWS Gpu machine : Looks like this line cannot fit in memory: X = np.zeros((len(sections), len_per_section, char_size)) --------------------------------------------------------------------------- MemoryError Traceback (most recent call last) in () 13 #Vectorize input and output 14 #matrix of section length by num of characters ---> 15 X = np.zeros((len(sections), len_per_section, char_size)) 16 #label column for all the character id's, still zero 17 y = np.zeros((len(sections), char_size)) MemoryError:

[@roopal_garg]

I have commented above on the same issue. See if that helps you.

[@funnychap9]

whts the ending track?

[@BenjaminSchultzLarsen]

can you start providing conda environments when you release your code, i am having multiple issues on python 3.5.3 and tensorflow 1.0.0 i did also have issues with your code in the music generation tutorial

[@simpleman5098]

have u tried docker?

[@BenjaminSchultzLarsen]

Simple Man no, but we have been told to use anaconda in the nanodegree. I will give docker a try

[@SirajRaval]

Thanks Ben for the feedback, i will try and handle dependencies better

[@napoleon_bonaparte2462]

Did Anyone else get this error on the loss computation? ValueError: Dimension 0 in both shapes must be equal, but are 25600 and 512 for 'SoftmaxCrossEntropyWithLogits' (op: 'SoftmaxCrossEntropyWithLogits') with input shapes: [25600,178], [512,178].

[@soorajrandhirsingh7480]

anyone?

[@soorajrandhirsingh7480]

fixed it just replace the line with loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits,labels_all_i)) with loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = logits,labels= labels_all_i))

[@napoleon_bonaparte2462]

I tried that, it didn't work. My error must be different than yours. I guess I will redo the whole thing.

[@revanttiwari4669]

hey siraj!! I am getting X = np.zeros((len(sections), len_per_section, char_size)) MemoryError please help!

[@payaspandey2842]

There are two ways you can do it, 1. Truncate the text ,i.e make it smaller so that it can fir your computer's memory text = text[:Size_that_fits] #Try many 2.Use a Generator if you want to use all text Stackoverflow will help

[@revanttiwari4669]

Payas Pandey thanks ...can you please elaborate on the 2nd method...I already did 1st one and am getting erraneous result

[@sukumarh3646]

It is taking too long...how can the code be made more efficient?

[@stumblinzz]

Neat. Nice job again

[@FalcoGer]

i see the link. it's some weird format though. how about some .py files?

[@bf2825]

Anyone got this? TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

[@kouohhashi]

Please use "outputs_all_i = tf.concat([outputs_all_i, output], 0)" instead of "outputs_all_i = tf.concat(0, [outputs_all_i, output])" for version 1.0

[@bf2825]

Thank you so much! it works now

[@vineetkothari2839]

wana learn about Random forest and its application.

[@Nola1222Piano]

Can i translate your code somehow to java. i work with that

[@RitobanRoyChowdhury]

I don't think there is a tensorflow wrapper for java, and its not forseeable in the near future because it would be much harder to perform some of these tasks in java (or any other statically typed or compiler based language)

[@DuhBroadcaster]

check out deeplearning4j

[@hunarahmad]

thanks for the awsome video

[@sukumarh3646]

If you have this error: TypeError: Expected int32, got list containing Tensors of type '_Message' instead. checkout this: stackoverflow.com/questions/41813665/tensorflow-slim-typeerror-expected-int32-got-list-containing-tensors-of-type

[@revanttiwari4669]

hey can you just point out what to do? i see documentation i get scared

[@kartikrajput2947]

change to outputs_all_i = tf.concat([outputs_all_i, output],0), initially it was outputs_all_i = tf.concat(0,[outputs_all_i, output])

[@AbgezocktXD]

ML Rap in doubletime is the best, yo

[@anahi6617]

hey broke have been following you and i an one of your true followers:-) and wanted to ask i want to go deep in machine learning as i want to make future in it, any suggestions, plz help me out.

[@SirajRaval]

thx Anahi, check out chris olah's blog for great material, much love

[@agustin9850]

Siraj thanks for your great effort and videos! Does someone get this error: IndexError Traceback (most recent call last) in () 1 for i, section in enumerate(sections): 2 for j, char in enumerate(section): ----> 3 X[i, j, char2id[char]] = 1 4 y[i, char2id[next_chars[i]]] = 1 IndexError: too many indices for array thanks in advance!

[@donviszneki1711]

I feel like this got off to a slow start and then Siraj starts pasting huge chunks of code and I'm like... "can we review the matrices he's defined again?" or similar things. I wish I could scroll while watching.

[@SirajRaval]

hm great feedback thanks

[@rushikeshdarge6115]

Can anybody summerize video in few lines, Please!!!

[@GG-Portal]

MemoryError Traceback (most recent call last) in () 13 #Vectorize input and output 14 #matrix of section length by num of characters ---> 15 X = np.zeros((len(sections), len_per_section, char_size)) 16 #label column for all the character id's, still zero 17 y = np.zeros((len(sections), char_size)) MemoryError: help me plz

[@parambhavsar1573]

try to reduce the length per section it may help you

[@napoleon_bonaparte2462]

Since the default datatype for np.zeros is 64 bit float and you're storing ints, you can try this: X = np.zeros((len(sections), len_per_section, char_size), *dtype = np.uint8*) y = np.zeros((len(sections), char_size), *dtype = np.uint8*)

[@donviszneki1711]

I think it would also be great if you had a partner to silently balance your brackets.

[@olfmombach260]

1:01:47 _"It looks like Latin"_ xD

[@懒猫精灵]

what something ?in your hand ,it is funny ,tell me, i want to buy it

[@andreas9109]

Generating fake news 😂😂😂😂😂😂😂😂😂😂😂😂

[@simpleman5098]

it's not fake, it's ALTERNATIVE :D

[@narutosaga12]

I really hope you're being sarcastic

[@chicken6180]

I know news, I know news very well, trust me they are FAKE NEWS. They have tremendous fakeness. They lie BIGLY. They are stealin our news jerbs. Build a wall around the fake news and make fake news pay for it. It will be tremendous.

[@SirajRaval]

haha. i had to say it

[@ganaraminukshuk0]

But you don't use a CNN to generate fake news, you use an RNN for that. CNN -> Convolutional Neural Network RNN -> Recurrent Neural Network Sorry, I had to.

[@GOOD__GOD]

Memory Error then Shape Error : this code is outdated

[@tunestar]

Do u really need voice of god?

[@NicolasBoa]

can you stop saying "from scratch" when it is not from scratch at all ? ...

[@SirajRaval]

yes good feedback