I love this man, and your keyboard sounds are the cherry on top.
@mildlyoverfitted7 ай бұрын
Hehe:) Thank you!
@blaine_stl3 ай бұрын
@@mildlyoverfitted what kind of keyboard are you using?
@DikkeHamster3 жыл бұрын
Cool to see the embedding evolution at the end!
@mildlyoverfitted3 жыл бұрын
Heh:) Glad you find it cool! Thank you!
@TheRovardotter3 жыл бұрын
At first, I did not understand the typing as sped up. Was very impressed by your typing skills.
@mildlyoverfitted3 жыл бұрын
Haha:D Just for the record I am not trying to make it look as if I was a fast typist. I speed up the typing parts because otherwise the videos would get ridiculously long:)
@TheRovardotter3 жыл бұрын
@@mildlyoverfitted nice vid by the way!
@navins22465 ай бұрын
Doing ML in vim is absolutely gigachad
@mildlyoverfitted5 ай бұрын
Hahaha:D
@vijayabhaskar-j3 жыл бұрын
Please tell me you're speeding up the portion of the video where you type, if not you're making others look like peasants in typing.
@mildlyoverfitted3 жыл бұрын
LOL! Of course I am speeding them up! ~3x when writing code, ~6x when writing docstrings. Otherwise the videos would get too long!
@namanjaswani89253 жыл бұрын
What an explaination !
@mildlyoverfitted3 жыл бұрын
Glad you found it useful!
@culturosityofficial2 жыл бұрын
Seeing the steps to how the embeddings worked under the hood got me through my assignment, thank you!
@kajalchaurasia9502 жыл бұрын
Just a thought Create a model for your videos. Input : Raw video Output : Normal speed where you are speaking and fast forward where you're typing.
@mildlyoverfitted2 жыл бұрын
Not a bad idea actually!
@fuat77752 жыл бұрын
Thank you for this video, very clear. BTW, you can use higer dimensions for embedding and use PCA to reduce into two dimensions when you need to plot
@mildlyoverfitted2 жыл бұрын
Thank you:) Yeh, great point:) I think I just wanted to make the video as short as possible:)
@IqweoR3 жыл бұрын
Very informational, definitely looking forward seeing your next videos!
@mildlyoverfitted3 жыл бұрын
Thank you! If you have any topic suggestions I would love to hear them!
@azmyin10 ай бұрын
Thank you very much sir. Deeply appreciate your hands on approach to teaching how to use embedding layer with LSTMs
@mildlyoverfitted10 ай бұрын
You are most welcome
@peter627093 жыл бұрын
Actually found this very helpful! The sped up typing sound was a bit annoying, but the content was great and it was well paced.
@mildlyoverfitted3 жыл бұрын
Thanks a lot for your feedback! Really appreciate it:) Yeh, the keyboard sound is something I also worried about. I will try to turn down the volume in my next videos, however, if you have other suggestions I would be more than happy to hear them:)
@randomforrest92513 жыл бұрын
@@mildlyoverfitted I actually enjoyed it a lot. Gave some hhkb asmr vibes
@connorsheehan45982 жыл бұрын
I think the keyboard sound is great, probably just a preference thing
@gnorts_mr_alien Жыл бұрын
@@mildlyoverfitted yah came here for the embeddings, stayed for the nice keyboard sounds
@LyurGG3 жыл бұрын
Okay but what keyboard setup are you using. Or is it a sound effect?
@mildlyoverfitted3 жыл бұрын
Actually, this is a result of speeding up the footage of me typing. I am using Adobe Premiere Pro.
@thepresistence59352 жыл бұрын
bro, what keyboard are you using. sound was amazing.
@mildlyoverfitted2 жыл бұрын
Logitec MX keys. Not a mechanical keyboard. The sound is augmented by the speedup effect.
@thepresistence59352 жыл бұрын
@@mildlyoverfitted Thanks a lot
@redone95532 жыл бұрын
Amazing quality! I'd love to know how you approach learning ML? Do you "just" visit courses at university and read math books? And what book/course would you recommend? Really appreciate an answer. Thanks!
@mildlyoverfitted2 жыл бұрын
It is a tough question because IMO everybody is different and has a different background. With that being said I learned some math, stats and basic ML+DL at my university. Since leaving the university, I have been working in the industry doing ML on a daily basis. Additionally, I have been working on hobby projects in my free time (e.g. this KZbin channel). So I would definitely recommend learning some maths first (calculus + linear algebra) and then learning ML+DL basics. Everything from then on should be as practical (=coding) and as regular (=getting an ML day job or coming up with ML projects) as possible. So IMO rather than studying an entire book or a going through an entire course it is better to work on a specific project and focus only on the theory/technologies that are needed for this project. I hope it helps but again everybody is different:)
@redone95532 жыл бұрын
@@mildlyoverfitted amazing! Really appreciate your long answer! Thanks, you helped me! Right now I study Bishop's famous book but your right, it's becoming boring and tedious and practical projects keep oneself motivated. Best greetings (:
@user-wb7mc7ip3z3 жыл бұрын
What's the keyboard? Sounds great when typing.
@mildlyoverfitted3 жыл бұрын
Logitech MX Keys. I think it is mostly the fact that my mic is right next to it. It's not a mechanical keyboard and it is not that "loud" in real life.
@mrigankanath73373 жыл бұрын
Hey great video!! liked and subscribed, Hey I really wanted if you could make a video on how to work with attention layers in Pytorch? AS you know they are very cool but there is lack of resources on how to use it in your custom model. It will be really great! Thanks!!
@mildlyoverfitted3 жыл бұрын
Thank you for the comment! That is a good idea:)))
@Danny-we4vz Жыл бұрын
Very thocky! What switches are you using?
@dddaaa9312 жыл бұрын
Thanks for your vedio i search a long time for this step and can not find a good answer
@mildlyoverfitted2 жыл бұрын
Thank you for your comment!
@locorocozhong3941 Жыл бұрын
dude your keyboard as weird typing sound, what keyboard are you using? Btw the video is nice
@mildlyoverfitted7 ай бұрын
Logitech MX Keys:) Thank you!
@trenvert123 Жыл бұрын
I liked the typing sound. I feel like the video is very information dense. I like it, but I'm having a very hard time parsing through everything. I think it's great for condensing this complex problem into a 20 minute video, when in reality, this video would have to be +1 1/2 hours to cover everything on word embedding alone. I think I'll come back to it at a later date when I have studied more. I'm still trying to understand how word embeddings work, and how word2vec like skip-gram allows them to represent similar words closer within the vector space.
@coc2912 Жыл бұрын
excellent
@ofeknourian89132 жыл бұрын
At 11:30 shouldn't you take the last layer of h instead of averaging over all of h? from my understanding the last layer of h represent the last prediction of the sequence.
@mildlyoverfitted2 жыл бұрын
Hey there! Great question. I guess taking the last layer of the hidden state is definitely a possibility. However, if the `n_layers=1` it does not make a difference.
@mikhaeldito2 жыл бұрын
Thanks so much for the video. Could you explain again how Embedding can accommodate unseen vocab? Thanks in advance.
@mildlyoverfitted2 жыл бұрын
Thank you. I guess the standard approach is to have a special entry in the vocabulary (like UNKNOWN) and whenever you stumble upon an unseen word/token/category you just use this special entry. The above strategy is exactly what BERT tokenizer does: github.com/huggingface/transformers/blob/6e535425feae20ca61a8b10ae5e8a7fab4d394ba/src/transformers/models/bert/tokenization_bert.py#L134-L136
@harshmankodiya93972 жыл бұрын
bro is rolling a dice at 10x speed
@mildlyoverfitted2 жыл бұрын
hahaha♥️
@beizhou24882 жыл бұрын
Are you interested in implementing reinforcement learning papers?
@mildlyoverfitted2 жыл бұрын
Sure, are there any that you find particularly interesting?
@ifthikhanmohamed1681 Жыл бұрын
text.txt what contains i got key error
@mildlyoverfitted Жыл бұрын
Hello! Can you maybe describe your problem in more detail?
@popamaji2 жыл бұрын
I didn't watch the video carefully but does it explain how nn.embedding code is been written or it's just the behaviour of nn.embeddings is explained in the first of video?
@mildlyoverfitted2 жыл бұрын
Great question:) This video shows the behavior and usage of `torch.nn.Embedding`. Its actual implementation is not discussed.
@popamaji2 жыл бұрын
@@mildlyoverfitted thanx for your awesome videos. can you make one for implementation, it would be great
@mildlyoverfitted2 жыл бұрын
That is an interesting suggestion! Thanks!
@The8thJester2 жыл бұрын
Thank you so much for this video! Do you perhaps have any suggestion on where my model might be going wrong? It seems that no matter what, my model always converges to outputting only space characters. I guess that's the most common character after all, but it's not quite what we're looking for is it? Thanks again!
@mildlyoverfitted2 жыл бұрын
Interesting! I would try to make sure that you are able to overfit on a small training set/a single batch of samples. This way you can be sure that the model has "enough power" to do the task. Then I would move on to training on all your data while monitoring your train/test loss and metrics! Hope that helps!
@iamragulsurya5 ай бұрын
Name of the font?
@mildlyoverfitted5 ай бұрын
So the theme I am using is here: github.com/morhetz/gruvbox . The README talks about the fonts I believe.
@saodeyipi96092 жыл бұрын
how can you type so fast, jz??
@mildlyoverfitted2 жыл бұрын
Heh, it is not my real typing speed. The video is sped up so that it is shorter.
@djfl58mdlwqlf2 жыл бұрын
are you coding with vim...? how is that even possible lol how can i get those codes beautifully painted
@mildlyoverfitted2 жыл бұрын
Yes. I am using this plugin: github.com/morhetz/gruvbox
@koiRitwikHai Жыл бұрын
Very informative video. Applause. I am just wondering ... why BERT does not use any norm in its embeddings? 1:40
@sumithhh93793 жыл бұрын
How the embedding weights are getting updated with a forward and backard pass. I understand the backprop but couldn't understand this. Please also reduce your typing sound. It feels like you are firing bullets instead or key strokes :).
@mildlyoverfitted3 жыл бұрын
Thank you for your comment! The weights are updated when the `optimizer.step()` is called and PyTorch takes care of the technical details. Regarding the typing sound, I actually turned it down in all of my recent videos!
@iraklimachabeli6659 Жыл бұрын
First of all thank you , very good intro combining embedding and LSTM. I run example and curious why in generate_text you use new_ch=np.random.choice( ..., p=probs) ,probs is logits[0] which is probability of next char being first element in vocab (which is ' ' blank char). I'd expect something like new_ch =dataset.ix2ch[ logits.argmax().item() ]
@mildlyoverfitted Жыл бұрын
Thank you for your comment! If I am not mistaken, `logits[0]` actually extracts the first and only sample from the batch. Or in other words, it is there to get rid of the batch dimension. Hope that helps:)
@AlokMalik182 жыл бұрын
I came from an internet search, without seeing like to dislike ratio I have no idea what's the quality of this video is. Please bring back dislike button youtube.
@mildlyoverfitted2 жыл бұрын
Totally agree! I hate the fact that they removed it.