Intro to Sentence Embeddings with Transformers

[@ax5344]

Hi, Thanks for explaining the SBERT model through clear visuals. The two pictures around @17:51 @18:19 are helpful to show the training process, but what I did not see is the SBERT model. Is the combination of the two visuals the SBERT model? That structure does not look like an embedding lookup. For new sentences, how can SBERT give out embedding? I did not see the new sentence embedding part present in the two visuals @17:51 and @18:19.

[@cameron.willis]

Thank you James! Very timely for our team! :)

[@jamesbriggs]

awesome - covering more on these tomorrow and next week :)

[@wiztech6563]

Clear explanation, thanks!

[@miguelgimenez3968]

Great Video James. I have one questions. I understand that during the fine-tuning the BERT weights are fitted to produce that sentences embeddings. But I don't understand the reason of needing a FFNN that takes [ u,v,|u-v| ] to compared both sentences. I suppose that it would be better to calculated the difference directly between two sentences and based on that difference decide the label ( neutral, contradiction...). With that the fine-tuning will be focus on changing the BERT weights and create a good representation of sentence to be compared instead of adding a FFNN that is not going to be used in the inference step of new sentences. I am sure that I have understand something wrong. Thank you for your help :)

[@jmparejaz]

great video James!

[@-cnio7803]

Thnx it would save me before I fall

[@ijaz6993]

I want to find similarities, clustering, topic modeling of small datasets and I focus on sentences similarity, so I should prefer sentence embeddings on Glove, Doc2Vec or any other embeddings techniques?

[@AnthonyCastrio]

What are LSTM's and GRU's? 1:35

[@braydenmoore3101]

Really great video. I’ve been comparing sentences just by distance (torch.dist). I’m not familiar with cosine similarity. What is the advantage?

[@flreview212]

Hello sir, thank you for providing an interesting lesson, I am a final semester student and very, very confused, my final project is text summarization, and have not gotten any results, do you have any advice on how sbert is used for text summarization or how can I do fine-tuning with my own dataset to get embedding generated by sbert? And all text using my own language not English :). Really appreciate it if you give me an answer, I'm really stressed right now, thanks in advance!

[@Sara-he1fz]

I am trying to retrieve the texts that have negative, or neutral sentiments from a pool of datasets and I have tested different queries, the results are not satisfying but I am wondering what would be a good query for this task? It seems this semantic search really depends on the query, and I am interested to know if there is any technique for this query generation for sentiment analysis?

[@freedmoresidume]

Great video, well explained. Thank you 🙏

[@MMetalRain]

So how does moving context vector to attention mechanism help with bottleneck? I was thinking analogy of database server vs stream of data. Many consumers can query database at the same time, but they are much less efficient if there is one stream that contains all data and they try to get something they need by reading all that data. But I don't really have clue if that makes sense in this context.

[@jamesbriggs]

That makes sense, I think of it like a pipe, via the attention mechanism the pipe is a lot wider, without it is very narrow. But I think your analogy is better as it includes the point that the decoder has access to and is retrieving those contexts

[@akashagrawal2754]

This is very helpful.

[@etherealshift9786]

is there any way to visualize the FFNN on this project using jupyter notebook. I kinda want to know what it looks like on code

[@jamesbriggs]

I think you should be able to extract FFNN weights and visualize with matplotlib heatmap or lineplot? Would be interesting I agree

[@etherealshift9786]

@@jamesbriggs can you make a video about neural networks too with semantic search :D

[@nitinat3590]

Thank you. You rock!

[@bilalsedef9545]

Great video Ben Fero! (A Turkish rapper) :)

[@jamesbriggs]

haha wow, I just need a few more days in the gym lol

[@MrCocochaneloo]

Good work

[@antoniomenta1327]

Great Video !!

[@shaminmohammed672]

Thank you

[@ricardocosta9336]

Holy fuck! Thank You! You rock!

[@jamesbriggs]

haha love the comment, thanks 😂

[@user-wr4yl7tx3w]

Audio could be better

[@michaelwechner3793]

Thanks for your explanations and demo! Do you have any experience with the OpenAI sentence/text embeddings beta.openai.com/docs/guides/embeddings/what-are-embeddings, e.g. api.openai.com/v1/engines/text-similarity-ada-001/embeddings ?

[@jamesbriggs]

Hey Micheal, actually looking into their embeddings now, I’m sure I’ll be posting something on them soon

[@ShivKatira]

@@jamesbriggs Are you Johnny Sins?