Let's Recreate Google Translate! | Model Training

[@shenchiang3837]

Thanks for making this fantastic video, would love to learn more about that model.generate function!

[@aswin0825]

I was wondering if you could help us with mc4 dataset in huggingface. Thanks for your effort! We really appreciate your efforts!

[@neoncommander]

Great videos!! Greetings from Spain, you have earned a new subscriber.

[@eah2886]

it is possible to make this model on vs code or any other IDE by tweaking the code a little bit?

[@梁大可-l5h]

Thank you!

[@jayrcastro10]

How do the model knows that it will convert english to japanese by just adding it as a token?

[@EdanMeyer]

That is because we format the training data that way. We can choose to do it in any other way we like. For example, we could start every sentence off with "Translate this from English to Japanese: ...", or we could use a completely random string of numbers to represent a translation pair. The important thing is that we are consistent throughout all the examples in our training and testing. So long as it is consistent, the model should learn the scheme we have come up with. The tags I used here could easily be changed and were somewhat arbitrary.

[@jayrcastro10]

@@EdanMeyer Understood! thank you so much for explaining it!

[@keburius2631]

hello, thanks for this tutorials, very helpful. I have 1 question. if i set model max_length to 256 or 512 before training. will the result model have opportunity to return translated text with this lengths? if not how can i get this result? thank you in advance

[@moseshu6917]

Thanks for making this fantastic video, Compared with facebook fairseq framework, which model is best?

[@titaniumiwnl]

How do you deal with unkown tokens? Or tokens which are out if vocabulary for the model

[@johndonne2170]

Hi, Edan and sorry for my English! Please suggest whether it is possible to create a model for a language for which there are few parallel translations (about 10,000 lines)? How effective will the model trained on such small data be?

[@dharaanv785]

Hey Eden!, we really appreciate ur efforts and it was a really useful but i'm just curious to try it with mc4 Dataset in huggingface can u help us with the same!

[@seyedmortezakamali2597]

Hi thanks for your content how can I make a convertor for programming languages for example python to c++

[@vanduytran5490]

thanks you sooo much

[@umarlawan5092]

nice project, i realy like it

[@charleenlozi4775]

thanks so much for this

[@dislike__button]

Awesome content! Will it work if you train the model on a dataset with languages different from the ones the model was pretrained on?

[@person-no2zs]

My question too. Have you figured it out yet?

[@EdanMeyer]

The bad news is that unfortunately it will almost certainly not work. The good news is that almost all the big language models have public checkpoints pretrained on multilingual data, so you can use one of those.

[@dislike__button]

@@EdanMeyer thanks for the answer. The problem is I want to train it on a low resource language that is not used in any ml models.

[@cupnoodles5956]

@@dislike__button I'm a year late but have you ever come up to a solution on this? If you don't mind I really wanted to learn and work on low resource language as well

[@dislike__button]

@@cupnoodles5956 I ended up fine tuning Meta's m2m100 model, replacing one of the low resource languages it was trained on (Zulu) with my own. It worked pretty well.