Rasa Algorithm Whiteboard - BytePair Embeddings

BytePair embeddings are a really cool idea. BytePair Embeddings can be seen as a lightweight variant of FastText. They need less memory because they are more selective in what subtokens they remember. This also makes them useful in certain scenarios because they can ignore subwords as well. They're also available in 275 languages!
If you want to see the Rasa NLU examples repo, go here:
github.com/Ras...
If you want to see the Whatlies repo for these embeddings, go here:
rasahq.github....
If you want to see the BPEmb repo, go here:
nlp.h-its.org/...

[@faangsde]

This channel is a gold mine! Thank you very much for sharing your insights!

[@87456100]

Great video! I think the video description lacks a word in the sentence "They need way memory..."?

[@distrologic2925]

they were masking that word to test your listening comprehension

[@distrologic2925]

How does the algorithm know when to stop merging tokens?

[@alanliang9538]

thanks bro, best explaination i can find.

[@wibulord926]

thanks for usefull tutorial

[@piyalikarmakar5979]

Thanks sir.. one query.. what's the difference between byte pair and wordpiece tokenization?

[@RasaHQ]

(Vincent here) Great question! My impression is that they are very similar in practice but that the way for merging letters is slightly different. I could be wrong but I think workpiece uses a likelihood heuristic while bytepair uses counts.

[@shahzadmalik96]

First comment

[@gorgolyt]

congratulations, you win nothing.