Sentence Transformers - EXPLAINED!

Let's talk about Recurrent Networks, Transformer Neural Networks, BERT Networks and Sentence Transformers all in one video!
Follow me on M E D I U M: towardsdatascience.com/likeli...
Joins us on D I S C O R D: / discord
Please like and S U B S C R I B E: / codeemporium
INVESTING
[1] Webull (You can get 3 free stocks setting up a webull account today): a.webull.com/8XVa1znjYxio6ESdff
REFERENCES
[1] The Main paper that introduced Sentence Transformers: arxiv.org/pdf/1908.10084.pdf
[2] The paper that introduced BERT: arxiv.org/pdf/1810.04805.pdf
[3] How BERT is different from transformers: ai.stackexchange.com/question...
[4] The triplet loss and architecture described in page 5: openreview.net/pdf?id=hga6dk7...
[5] AWS K-NN elastic Search: docs.aws.amazon.com/opensearc...
[6] Approximate Nearest Neighbors algorithm used by spotify (ANNOY): github.com/spotify/annoy
[7] Colah’s blog that very well breaks down LSTM networks: colah.github.io/posts/2015-08...
[8] My video on LSTMs, explaining what you need to know: • LSTM Networks - EXPLAI...
[9] My original video on Transformer Neural Networks: • Transformer Neural Net...

[@CodeEmporium]

Hey Everyone! Hope you're all doing super well. This video will give you everything you need to know about Transformer Neural Networks, BERT Networks and Sentence Transformers - or at least all that we can cover in 17 minutes. Hoping we all understand why these Architectures were developed the way they were, painting the picture as a fluid story. I'm trying another teaching style here. If you like this kind of video, please do let me know in the comments. Put a lot of effort into this, so I hope you think this is good! Enjoy! And Cheers!

[@LiaAnggraini1]

Thank you! This is what I need for my thesis

[@manash.b4892]

Wow. Thanks a lot for all these videos. I am self-studying beginner and your videos have been a boon. Keep up the good work, man!

[@prasannabiswas2727]

Really the best info out. Thank you.

[@HazemAzim]

really neat . Thank you , I was looking for nice stuff on SBERT with decent depth

[@Daniel-gy1rc]

dude you are amazing. Hope you keep this work up! Explaining complex things in an easy-to-follow and examplified way is a great skill!

[@CodeEmporium]

Thanks a ton Daniel! Much appreciated complements :)

[@simoneparvizi775]

Hey man huge fan! Would you do a video about the "vanishing gradient problem"? Tbh I've been looking for a good video on it, but they're just not on point as you are....I'd really like your explanation on such argument! Keep up with the great work

[@kevon217]

Excellent overview!

[@TheHamoodz]

This channel has orders of magnitude more views than it deserves

[@RaghavendraK458]

Great video. Thanks

[@WhatsAI]

Amazing overview !

[@user-vu4or4ih8p]

Great stuff. Thanks

[@JJ-dz2ne]

Very informative, thank you!

[@CodeEmporium]

You are very welcome! Thanks for watching and commenting

[@masteronepiece6559]

Nice overview

[@TheShadyStudios]

Great choice!

[@freedmoresidume]

Great video, thanks a lot

[@CodeEmporium]

Welcome:)

[@moslehmahamud9574]

this is good!

[@clairewang8370]

This is 🔥!!!😍😍😍😍😍

[@PritishMishra]

Great video!!! Can we get some project videos on Transformer? As you showed in this video about the text-similarity with BERT so do you have any plan to create a video to do this with python?

[@thekarthikbharadwaj]

Yes, really needed. Internet is lacking with an exact project developed using Transformers with proper backend information

[@miriamramstudio3982]

Great video. One part I didn't completely understood is the NLI part. Do you mean that after that NLI step, the mean pooling sentence vector of the newly trained BERT not be "poor" anymore? Thanks.

[@NicholasRenotte]

Oooooooh, this is so freaking cool!! When are we teaming up to build something?!

[@CodeEmporium]

Dude. I will reach out ma guy (sorry i didn't before) :)

[@NicholasRenotte]

@@CodeEmporium ayyy no problemo man!

[@keerthana2354]

Can we use this for comparing two web articles?

[@kestonsmith1354]

My favourite model to train is T5. So much better . I don't like encoder models . I rather use a model that uses both encoder and decoder rather than either/or.

[@shoukatali5671]

Приятно

[@Han-ve8uh]

Could you explain these 2 points in more detail? 3:21 transformers weren't designed to be language models + 16:35 transformers not complex enough to train a language model 1. What are language models supposed to do that transformers can't? My interpretation is that transformers do seq-seq tasks like translation, and translation needs a language model, so transformers are language models. Anything wrong with this thinking? 2. Can I say transformers are only invented to parallelize RNN family of models with attention? Any other obvious general or task specific benefits of transformers?

[@mizoru_]

I guess he means that they get better through improved pretraining (thus understand language better) From Papers with code: "BERT improves upon standard Transformers by removing the unidirectionality constraint by using a masked language model (MLM) pre-training objective. The masked language model randomly masks some of the tokens from the input, and the objective is to predict the original vocabulary id of the masked word based only on its context."

[@Slayer-dan]

Thank you sir.

[@norlesh]

5:11 Bidirectional Encoder Representation FROM Transformer (not of Transformers)

[@InquilineKea]

QUORAAAAA

[@CodeEmporium]

AAAHHH

[@roccococolombo2044]

It is spelled chien nor chein