OpenAI Whisper: Robust Speech Recognition via Large-Scale Weak Supervision | Paper and Code

[@TheAIEpiphany]

Let me know whether the code part helped! :) Is it adding any value for you guys? Or am I just rambling and it's too hard to follow unless you play with the code yourself? Would really appreciate some feedback!

[@xl0xl0xl0]

It definitely did! Is the debugger your first choice when it comes to figuring out how some new codebase works, or did you fire it up for the occasion as a demonstration tool?

[@TheAIEpiphany]

@@xl0xl0xl0 thanks!! I have a whole series where I do just that. And as for your question - it depends. If I am playing with something on my own then yes, always! By far the best way to understand every single detail of your code

[@Erosis]

@@TheAIEpiphany I missed that series! I actually struggle with debugging ML code with vscode, so I'll check it out!

[@leobeeson1]

This code walkthrough has made this paper walkthrough one of the best I've seen. Thanks for that, and please keep doing it!

[@TheAIEpiphany]

@@leobeeson1 wow nice, thanks for telling me that! If I get more feedback I might keep doing this in every paper walk through!

[@mariatrofimova5512]

Thanks for walking through Whisper code together, enjoyed the journey!

[@pratikkhedikar6759]

mmmaaannnnn !! What a good video. I was like searching for something like this. Where in even a noob like me can understand the entire paper because you took through it step by step! I knew this was going to be a great video when you stopped to explain log-mel spectrum as well! Thanks Aleksa

[@devhau5]

I just found this channel and I’m SO THANKFUL for a great walkthrough and explanation. It’s super fun. This is gold!!! Thanks Aleksa!

[@alexgil55ka]

This is super cool man! Thanks for diving deep into it

[@TheAIEpiphany]

Thanks Alex!

[@DevasheeshMishra]

Loved it.. need more such videos

[@DevasheeshMishra]

Rewatching the stream

[@huonglarne]

Thank you so much for doing these videos. You helped me so so so so much.

[@FreeSubtitlesAI]

Very informative and authoritative, thank you!

[@Spockleblupit]

Thanks Aleksa! Really appreciate the effort you put into this videos. Quality content, keep it up.

[@convolutionalnn2582]

Sir,I have read your roadmap to Reinforcement Learning...I wanna do research in RL...1)Should i still follow your roadmap ? 2) Do i need to know the whole maths derivation behind Supervised Unsupervised and Deep Learning Algorithm 3) How can i start doing research in RL in undergraduate in an non research institute?

[@pocco8388]

Thanks for making this great video!

[@CHENXIN-pn7oh]

So well explained! Thx!

[@nnpy]

Great video!!

[@petercowling6769]

Welsh an outlier. Never would have guessed. Anyway, gotta go, heading out to Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch this afternoon.

[@ludvigjoborn7937]

One can find in the paper that this is because a lot of Welsh was misclassified as English during the data collection process. Imagine them finding out.

[@curryeater259]

You are amazing sir!

[@kerenstarobinski8564]

is it possible to find the timestamps of each transcribed word? Great work!

[@MyHowHowHow]

Not in OpenAI's version but a fork of it has this feature. It is called WhisperCpp

[@JF-vt4ve]

impressive work!

[@amilia4174]

I have watched your video and it was great! But I'm not sure whether the translation and transcription tasks share the same decode parameters.

[@ChuanChihChou]

I wonder if we can use the attention map (of how much each audio token contributes to the prediction of each transcript token) to back out timestamps instead?

[@MattUebel]

This is great, ty!

[@TheAIEpiphany]

🚀

[@vinayakbaddi29]

@Aleksa Gordic, Thanks for sharing this valuable information. Apart from AI would look to see how you are using VS code so effectively to move between the code and debug it. Would really appreciate it if you could provide more information on the same on video.

[@goryeodynasti3025]

@TheAIEpiphany , how do you see the effect of "best_of" parameter in the quality of the transcription? Any insight would be helpful. Thanks

[@asceznyk]

Hi Aleksa! Great video! I just wanted to know what would the loss function be for the models? Would it be something like cross-entropy? Because the model predicts tokens..

[@sibadattasasmal4866]

can you put little emphasis on how the time stamps are generated for transcription

[@iradaaristova8698]

can you make a video how decoder works?

[@chandanabandaru7386]

hey i am using whisper model for speech to text convert but i am getting tensor error for large audio files (30 min to 1 hr). can anybody help me ? i am testing with postman

[@MoiezIzmail]

it's not chinese, it's korean(script is called han-gul). Thanks for the tutorial!

[@xXMaDGaMeR]

can the model be ram locally? how much computing to run this model for inference

[@kshitizkhandelwal9348]

Can someone explain how are embeddings learnt?

[@lavkushdas5529]

hey can you provide your source code that u have written in vscode?

[@FinnBrownc]

Would be helpful if you could put these models in history a bit. I’m not as familiar with how things were done in the past vs. today SOTA.

[@tahercoolguy123]

Hey really nice video. Can we fine tune whisper model for our dataset. If yes can you show us how

[@phongtranhung5635]

Hi. I've found you channel and the videos are totally all mind blowing. I have a question regarding Whisper. Currently I want to return a list of all transcribed words probability. I think that I have something to do with the def update inside Decoding.py. Can you make some help on how to do it? I would be very appreciated!

[@huonglarne]

You can modify the update function to return the logprobs of all words. The max of that logprobs is the selected token's probability.

[@marearts.]

1:30 yes it's korean

[@dimorischinyui1875]

Hey guys please can anyone help me with this issue. I am trying to run whisper on my machine and I am getting this error in cmd. UserWarning: FP16 is not supported on CPU; using FP32 instead warnings.warn("FP16 is not supported on CPU; using FP32 instead"). I use a windows 10 with gpu RTX2060. Also it seems it runs on my cpu instead of NVIDIA GPU. I created a python virtual environment and pip installed whisper in that virtual environment just for more details.

[@MyHowHowHow]

Try the parameter --device cuda

[@HarishPentapalli]

Any guesses on the name of company B?

[@TheAIEpiphany]

Hah, hard to infer without repeating the research

[@BRICKSINSILK]

yes all well and good. but make it not hold onto a timestamp for ten years and then halucinate

[@SuperMan-rw6iz]

That's not even Mandarin... it's Korean BTW 😅

[@FinnBrownc]

Would be helpful if you could put these models in history a bit. I’m not as familiar with how things were done in the past vs. today SOTA.

[@TheAIEpiphany]

Thanks for the feedback, you mostly care about transformers here. "Attention is all you need" paper