Sentence Transformers - EXPLAINED!

Let's talk about Recurrent Networks, Transformer Neural Networks, BERT Networks and Sentence Transformers all in one video!
Follow me on M E D I U M: towardsdatasci...
Joins us on D I S C O R D: / discord
Please like and S U B S C R I B E: / codeemporium
INVESTING
[1] Webull (You can get 3 free stocks setting up a webull account today): a.webull.com/8...
REFERENCES
[1] The Main paper that introduced Sentence Transformers: arxiv.org/pdf/...
[2] The paper that introduced BERT: arxiv.org/pdf/...
[3] How BERT is different from transformers: ai.stackexchan...
[4] The triplet loss and architecture described in page 5: openreview.net...
[5] AWS K-NN elastic Search: docs.aws.amazo...
[6] Approximate Nearest Neighbors algorithm used by spotify (ANNOY): github.com/spo...
[7] Colah’s blog that very well breaks down LSTM networks: colah.github.i...
[8] My video on LSTMs, explaining what you need to know: • LSTM Networks - EXPLAI...
[9] My original video on Transformer Neural Networks: • Transformer Neural Net...

[@CodeEmporium]

Hey Everyone! Hope you're all doing super well. This video will give you everything you need to know about Transformer Neural Networks, BERT Networks and Sentence Transformers - or at least all that we can cover in 17 minutes. Hoping we all understand why these Architectures were developed the way they were, painting the picture as a fluid story. I'm trying another teaching style here. If you like this kind of video, please do let me know in the comments. Put a lot of effort into this, so I hope you think this is good! Enjoy! And Cheers!

[@Daniel-gy1rc]

dude you are amazing. Hope you keep this work up! Explaining complex things in an easy-to-follow and examplified way is a great skill!

[@CodeEmporium]

Thanks a ton Daniel! Much appreciated complements :)

[@Han-ve8uh]

Could you explain these 2 points in more detail? 3:21 transformers weren't designed to be language models + 16:35 transformers not complex enough to train a language model 1. What are language models supposed to do that transformers can't? My interpretation is that transformers do seq-seq tasks like translation, and translation needs a language model, so transformers are language models. Anything wrong with this thinking? 2. Can I say transformers are only invented to parallelize RNN family of models with attention? Any other obvious general or task specific benefits of transformers?

[@mizoru_]

I guess he means that they get better through improved pretraining (thus understand language better) From Papers with code: "BERT improves upon standard Transformers by removing the unidirectionality constraint by using a masked language model (MLM) pre-training objective. The masked language model randomly masks some of the tokens from the input, and the objective is to predict the original vocabulary id of the masked word based only on its context."

[@brewingacupofdata]

Great Video! One minor comment: Shouldn't the loss equation from the triplet method (13:58) be the other way around? The difference that is subtracted should be between the anchor sentence and the negative sentence.

[@PritishMishra]

Great video!!! Can we get some project videos on Transformer? As you showed in this video about the text-similarity with BERT so do you have any plan to create a video to do this with python?

[@thekarthikbharadwaj]

Yes, really needed. Internet is lacking with an exact project developed using Transformers with proper backend information

[@MohammadShafkatIslam-k4x]

Great video. I just figured out the issue with my dataset, after I had bad results from directly using Roberta

[@kestonsmith1354]

My favourite model to train is T5. So much better . I don't like encoder models . I rather use a model that uses both encoder and decoder rather than either/or.

[@roccococolombo2044]

It is spelled chien nor chein

[@simoneparvizi775]

Hey man huge fan! Would you do a video about the "vanishing gradient problem"? Tbh I've been looking for a good video on it, but they're just not on point as you are....I'd really like your explanation on such argument! Keep up with the great work

[@NicholasRenotte]

Oooooooh, this is so freaking cool!! When are we teaming up to build something?!

[@CodeEmporium]

Dude. I will reach out ma guy (sorry i didn't before) :)

[@NicholasRenotte]

@@CodeEmporium ayyy no problemo man!

[@BinayGupta-ny7bp]

Great explanation dude

[@nadavnesher8641]

Brilliant video 🚀

[@safwanmohammed7715]

Masterpiece 💯

[@LiaAnggraini1]

Thank you! This is what I need for my thesis

[@norlesh]

5:11 Bidirectional Encoder Representation FROM Transformer (not of Transformers)

[@miriamramstudio3982]

Great video. One part I didn't completely understood is the NLI part. Do you mean that after that NLI step, the mean pooling sentence vector of the newly trained BERT not be "poor" anymore? Thanks.

[@manash.b4892]

Wow. Thanks a lot for all these videos. I am self-studying beginner and your videos have been a boon. Keep up the good work, man!

[@WhatsAI]

Amazing overview !

[@HazemAzim]

really neat . Thank you , I was looking for nice stuff on SBERT with decent depth

[@TheHamoodz]

This channel has orders of magnitude more views than it deserves

[@keerthana2354]

Can we use this for comparing two web articles?

[@prasannabiswas2727]

Really the best info out. Thank you.

[@RaghavendraK458]

Great video. Thanks

[@GeoffLadwig]

Great stuff. Thanks

[@kevon217]

Excellent overview!

[@clairewang8370]

This is 🔥!!!😍😍😍😍😍

[@moslehmahamud]

this is good!

[@InquilineKea]

QUORAAAAA

[@CodeEmporium]

AAAHHH

[@TheShadyStudios]

Great choice!

[@Slayer-dan]

Thank you sir.

[@JJ-dz2ne]

Very informative, thank you!

[@CodeEmporium]

You are very welcome! Thanks for watching and commenting

[@masteronepiece6559]

Nice overview

[@shoukatali5671]

Приятно

[@freedmoresidume]

Great video, thanks a lot

[@CodeEmporium]

Welcome:)