Lecture 12: Recurrent Networks

[@chriswang2464]

Mad respect to Justin!

[@neerajkrishna1983]

The gradient flow and introduction to LSTM was great!

[@roboticseabass]

Another common RNN trick worth mentioning is bidirectional RNNs. So basically you have 2 independent RNN layers -- one that goes through a sequence forwards and another backwards -- and you concatenate their hidden layer outputs at the end. If you have full sequences available this can help!

[@mostafashahhosseini3378]

I wish Justin could teach any topic in the world

[@aritraroygosthipaty3662]

42:05 Justin goes on to say that the color blue represents all off, but in his paper, it is quite clearly mentioned that -1 is red and +1 is blue. Another thing to ask here is, the explanation of the color text is reasonable, but in the paper, it states that the text color corresponds to tanh(c). Are we looking at the hidden states of the LSTM or the memory state?

[@eddie31415]

Thanks a lot!

[@kainatyasmeen5608]

Great learning. Thanks alot!

[@MrAmgadHasan]

image Captioning 43:42

[@HesitantOne]

at 27:09 shouldnt embeddings of last 2 input equal? they are both same token. why their embeddings are different?

[@mostinho7]

Start at 13:00

[@taghyeertaghyeer5974]

@32:00, I am wondering, why did Justin say: "Once you process one chunk of data you can throw it away, evict it from the memory, because all the information needed for training from this chunk is stored in that final hidden state of the RNN at the end of processing the chunk". I guess the data from this chunk is saved in all the hidden states obtained at the end of processing the chunk. Am I correct?

[@itchainx4375]

probably not, just the last output of this trunk

[@thinhvu6902]

It should be the last hidden state obtained at the end of forward processing the chunk