Word Embedding in PyTorch + Lightning

[@statquest]

Here's the code: lightning.ai/lightning-ai/studios/statquest-word-embedding-with-pytorch-lightning?view=public&section=all To learn more about Lightning: lightning.ai/ Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@loserc1854]

waiting for ur next book

[@drintro]

The best part of this example is the way the first example is written with single dimension arrays for all of the parameter values. That makes the matrix operations explicit and more clear even to an experienced developer. I recommend writing and debugging the code from watching the video. There are small things that you might get wrong that will teach you something.

[@statquest]

Thank you! I'm glad you liked the example.

[@ijf03208rek]

I bought The StatQuest Illustrated Guide to Machine Learning, and it's absolutely amazing, same with every single one of your videos! I highly recommend that book to anybody who is interested in learning about this stuff! Thank you thank you thank you so much for taking the time to put together the content you make, I can't believe how well you paint this stuff into perspective! You're a freaking awesome person Josh, I'm still absolutely elated that I stumbled across this channel! I never would have expected to be so consumed with learning this and everything about this!

[@statquest]

Thank you very much! I'm really glad you enjoy my videos and my book! Thank you! :)

[@vishnumuralidharan9858]

Hi Josh, I just want to say that you are an absolutel godsend to the ML and DS community. I have been following your content since 2020 and you have never let me down. I hit a block with PyTorch code implementation and your videos simply untangled a lot of my mental knots. Keep up the great work!

[@statquest]

Awesome! Thank you!

[@InglesConConfianza]

Just finished the Deep Learning playlist. Thank you so much for this great work.

[@statquest]

BAM! Thank you very much! You deserve an award!

[@exxzxxe]

Josh- you are a Master in making the difficult comprehensible!

[@statquest]

Wow, thanks!

[@tom19860526]

I like your videos. Both the slides and explanations are very detailed and clear. I cherish them every time I watch them. It is a good learning video. thank you very much.

[@statquest]

Thank you very much! :)

[@beegdigit9811]

Your content is really for the future generations

[@statquest]

Thanks!

[@itaydagan7459]

Josh you are one of a kind!! Thank you for all the content!

[@statquest]

Thank you!

[@itsawonderfullife4802]

I too, love your work and specially appreciate your playful style and all the movie references. ;)) Thanks for your videos, all of them.

[@statquest]

Thank you very much!

[@bakerb-rz6lv]

I have a bug when I import the numpy: ImportError: numpy.core.multiarray failed to import. My python is 3.10, while it works in python 3.8. In python 3.10 I can not degrade numpy to 1.20 or lower, maybe some module did not compatible.

[@statquest]

Try running it on lightning and see if it works there.

[@nourin_siddique]

Yayy !! I was wanting a video like this !!!

[@statquest]

BAM! :)

[@marwolaeth111]

StatQuest is so inspiring! Please consider making a 'Quest about Geometric deep learning. What do you think?

[@statquest]

I'll keep that in mind.

[@eliyahubasa9401]

Great video, a great way to study. Thank you.

[@statquest]

Thanks!

[@RaynerGS]

You rock. I love your work. Salute from Brazil.

[@statquest]

Muito obrigado! :)

[@elodiebeitman8251]

Hi! I also bought The StatQuest Illustrated Guide to Machine Learning - It's awesome! Triple BAM! I think there might be a small error with the formula on page 135 (Naive Bayes: FAQ Part 3) though, FYI! Thanks again!

[@statquest]

This is a known error in the book. All of them are listed here: statquest.org/sigml-errata/

[@TJ-hs1qm]

StatQuest Time!!

[@statquest]

BAM! :)

[@abhilashb722]

This is really great 🎉🎉🎉. Can you please continue this pytorch + lightning series

[@statquest]

Yep! Working on it right now.

[@kisholoymukherjee]

And hope you will also add them to the Deep Learning Playlist. Keep rocking (and BAM-ing!) @@statquest

[@statquest]

@@kisholoymukherjee Will do!

[@diegoandradex12]

Great video as always

[@statquest]

Thanks again!

[@sweetlearning6629]

Thing is; i always look forward to new videos from you, cause i know it would be awesome as always. I would want to see more videos on computer vision tho, just hope you'd consider this. Thanks

[@statquest]

I'll keep that in mind.

[@gsestream]

as a related thing, finding closest things on a map/grid, without making a cell division system, sort all dimensions then check if they collide, like spheres or aabb bounding boxes, n log n time complexity, as opposed to n^2 time complexity, similar to grid cell partitioning of data, for clustering, or k-nearest clustering, but just finds closest at range, any number, very fast. should be applicable directly in high dimensionality complex data analysis, dimensionality reduction, covariance matrices, yep its designed to be fast

[@mohamedthasneem7327]

Thank you very much for the video sir.

[@statquest]

Welcome!

[@vigneshvicky6720]

Plz start teaching yolov8 which is used for object detection, segmentation, classification ... Every problem will be solved plz plz

[@statquest]

I'll definitely keep that in mind! Keep reminding me on future "in PyTorch + Lightning" videos.

[@vigneshvicky6720]

@@statquest sure sir but try to do it from scratch because everyone are using build in one so many of them dont know what is going inside. I would like to learn it from scratch so that I can finetune architecture myself . Do it as soon as possible🙏

[@d25102]

Thank you!

[@statquest]

:)

[@VishalKumar.S-g8f]

Hi josh Starmer, after learning ML, whether this series is enough for a s beginner to learn deep learning? and do any additional things to know other than this series for deep learning?

[@statquest]

It's a great way to start.

[@yashsonune4391]

Thank you again for the quality content. 🔥. Btw is there any plans for teaching about fine-tuning Language models. I heard this paper (Universal Language Model Fine-tuning for Text Classification) is of higher relevance and backbone for many state of the art solutions.

[@statquest]

I'm working on one.

[@kartikchaturvedi7868]

Superrrb Awesome Fantastic video

[@statquest]

Thanks!

[@RaynerGS]

The original work [1] uses a multidimensional vector for each word instead of a unique real value. For instance, Troll2 = [,025, 0,735, 0,256, 0,145], in this case, four dimensionality. In the paper, the authors use a matrix instead of a neuron. Question: using neurons, how would you increase the word dimensionality representation? 1: "Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space. In Proceedings of Workshop at ICLR, 2013."

[@statquest]

The number of values associated with each word is determined by the number of activation functions in the hidden layer. If you want 4 numbers, then add 4 activation functions.

[@horrorislander]

At 25:30, hidden = self.input_to_hidden(input) but at 26:17, hidden = self.input_to_hidden(input[0]) Similiarly, at 25:48, loss = self.loss(output_i, label_i) but at 26:17, loss = self.loss(output_i, label_i[0]) Note the absence/presence of a subscript. Does this matter? My "type-along-with-Josh" code had problems when I ran it, which is why went back and checked. When I adopted the code from the "small-font" version at 26:17, it seemed to run correctly, but since I know nothing more about these modules other what's in your videos... maybe it doesn't matter? There certainly could be some other issue with my "type-along" code, maybe one I inadvertently fixed or randomly didn't break the program the second time. Still, I thought it worth mentioning the differences. Thank you for the video series!

[@statquest]

I'll be honest, I thought the code at 26:16 was so small that no one would ever actually read it - but I still apologize for the confusion it caused. That being said, it's always better to download the code rather than type it in, because I actively maintain it, and when the library changes, I'll change the code there (I can't change the video, which is a bummer).

[@DumplingWarrior]

At 11:45 when we set input = input[0], we're only selecting the input that corresponds to predict what comes after Troll 2. Does that mean if we'd like to predict what comes after "is" with input [0,1,0,0], we'd need change this code to input[1]? Seems oddly inconvenient to have to change something in a class for different scenarios.

[@statquest]

No. All "input[0]" does it remove an extra set of brackets. We get the same extra set for each row in the dataset, so we don't have to change this code.

[@d_b_]

👍So, these are word embeddings. Do you know/think that sentence embeddings and text embeddings differ much from this process?

[@statquest]

I'm pretty sure they are the same, but I'm not certain.

[@lebronjames8507]

Do u guys have an education I can get somewhere? I saw u guys had a probability basics playlist but do u have some type of course to take me through every single subtopic of stats?!

[@statquest]

Not yet.

[@debarunkumer2019]

Can you please create a playlist to demonstrate the coding part of Encoders, Decoders and Transformers? This is an earnest request from your FAN. Thanks.

[@statquest]

I'm working on it right now.

[@deepikasheshabutter4790]

When is the next video comingggg???

[@statquest]

It just came out: kzbin.info/www/bejne/eWq0hKOiatOgqLs

[@Johan-zs9xh]

Excuse me, what is your code editor?

[@statquest]

I use jupyter.

[@iurgnail]

do a video for ARIMA and VAR and their cousins please!

[@statquest]

I'll keep that in mind.

[@mikinyaa]

🥳

[@statquest]

YES! :)

[@arseniykan]

BAMbastic

[@statquest]

Thanks!

[@wibulord926]

what is the different between pytorch and pytorch lightning sir ?

[@statquest]

PyTorch Lightning is something that works with PyTorch that makes it easier to code and makes it easier to scale in the cloud and makes your code run faster in genernal.