2. OpenAI Whisper - Fed Speech Recognition

I will be starting a spinoff channel on AI in music, art, and gaming in 2023. Subscribe at: / @parttimeai
In this video, we learn how to use OpenAI Whisper for speech recognition. We write Python code to transcribe Fed Chairman Powell's November 2nd speech.
Source Code:
colab.research...
gist.github.co...

[@parttimelarry]

Like the video? Support my content by checking out Interactive Brokers using the link below: www.interactivebrokers.com/mkt/?src=ptlPY1&url=%2Fen%2Findex.php%3Ff%3D1338 I will be starting a spinoff channel on AI in music, art, and gaming in 2023. Subscribe at: youtube.com/@parttimeai Source Code: colab.research.google.com/drive/1vJj6EW0nKi3olLb5nOccA4ypkKx-x4Hr#scrollTo=xUot3GYB64tv gist.github.com/hackingthemarkets/c6ca7834d2af4932e3ab0d847679c14e

[@sriramkrishna6853]

Hi Larry, what’s the cost of using GPU’s with Google colab? Is there a reason you picked that? Will be great if you share some thoughts around it.

[@itagakixxx]

hello, can you tell me how AtheneWins (youtube name) make his AI streamers ?

[@justtohost3431]

Makes my blood pressure go crazy - so many great possibilities! This is insane, unique! One-time chance to contribute to this world in a unique way!

[@LindsayHiebert]

Outstanding OpenAI Whisper demonstration! Well done! Great job! Thank you.

[@buffaloBean]

Another great job Larry. Love the enthusiasm, curiosity and display if patience in your content. A natural teacher!

[@buffaloBean]

Has anyone tried using chatgpt to enhance their programming?

[@asbjrnthorsen7457]

I got really inspired watching your videos! Great job!

[@jerrywang3225]

The implication of this application is beyond our imagination. Thanks for this great content.

[@jayhu6075]

What a amazing tutor. I like your style to make it understandable for everyone. Many Thanks.

[@danielattard]

Larry - A sincere thank you for teaching us all about how to use machine learning models like OpenAI Whisper. In particular, I also learned a lot from the Google Colab which was incredibly helpful. 👍 Can't wait to see what you build next.

[@akzorz9197]

Was just about to say, you could now wire this into a MetaTrader EA for making decisions on its positions, etc. As 2 minute papers would say - "What a time to be alive!"

[@NeferLopez]

I'm learning so much from you, Larry. Thank you so much!

[@breakoutgaffe4027]

incredible stuff mate! thanks

[@loryo80]

thank you Larry for all informations and for your pedagogy, i love your videos

[@phil.d6449]

Larry, thank you for all your hard work. I'm really looking forward to seeing the next videos!

[@gljames24]

I was looking to transcibe some of my audio for script writing. This looks great.

[@GrizzCodes]

Thank you Larry for the priceless information you provide everytime!

[@OptimusPrime-n5o]

Fantastic, Larry👌💯 TYVM bro. 🤝

[@bakari007]

Excellent approach ❣ ! Thank you for sharing ! More interesting could be scanning if there's any speech available and live stream interacting :D !! Thank you for the video !

[@quangdinhdota2388]

thank you so much. This video is amazing

[@ronaldronald8819]

Thats interesting. I am not into finance but what you showed is applicable to a ton of stuff. Wish i had more time.. So many interesting things going on.

[@MrEvansjethro]

Thank you so much, Larry.

[@andydataguy]

Excited for the series!! Would love to see something that writes in depth research reports from a mix of DaVinci, web scrapers for non-intuitive data, and ChatGPT for output and language finesse

[@sriramkrishna6853]

Looking forward to more OpenAI videos from you, Larry. Gonna spend sometime watching and learning. Cheers!

[@davidbarth80]

Interesting! Well done. I believe you could take keywords from the speach and if these keywords appear in upcoming speaches you could open positions based on that.

[@michakupis6198]

If anyone is getting an issue retrieving the YT video data using pytube, you will need to authenticate by using this code: youtube_video = KZbin(youtube_video_url, use_oauth=True, allow_oauth_cache=True) it will ask you to enter a code visible in the terminal, log in to YT and then you should be good to go.

[@SneyDeag]

In which part of the code does it go? Could you help me? I have this same problem

[@trsch5384]

Incredible work! Thanks a lot

[@blakesamic]

Great video. Thanks Larry!

[@WallStreetPapi-nx8he]

Thank you Larry!

[@sealt00th]

that was really interesting to watch! thank youu

[@HenriqueSardinha]

Great content! Would be great if you could follow up in another video with action taken from your analysis. I.e run the models to decide what to do with that information and compare it with what really happened

[@trainspotting02]

Great work! 🤗

[@ControlTheGuh]

Great content again mate!

[@pietrusso]

This is phenomenal, thank you!

[@TradingLaboratory]

Love the content Larry

[@blender_wiki]

Impressive and scary 😜. Very nice approach.

[@j-5474-k9z]

sweeeet! gonna be auto-trading futes by the next fomc!!!

[@skeletonmasterkiller]

Really love your commitment to building in public for all of us. Thanks for all of your efforts do you have a patreon so I can support you for your efforts.

[@parttimelarry]

Hey thanks! I don't have a patreon, but there is a buymeacoffee link on my channel page buymeacoffee.com/parttimelarry

[@aiautoglasscrm]

Awesome video, even though I am not into finance. Subscribed

[@krissn8111]

Very impressive

[@LeveragedFinance]

make more cool videos like this I learned a lot

[@cnaccio]

This is super aweome!

[@tak68tak]

amazing. can't wait for the next tutorial.

[@Robay146]

Thanks for this!

[@shephusted2714]

it will be so good to see significant forks - models will get better and the tech keeps advancing so they will try to go more for profit but the masses may be able to spawn competitors - things happen

[@maxiperelli8570]

Great video

[@code-grammardude5974]

Very clear tutorial ty

[@SutboxSutty]

Go Larry your a badass....Respect

[@ShadedWarrior]

You're awesome. Thank you!

[@machdragon]

Been using OpenAI a bunch lately as well. Pretty useful for figuring out regex or sql queries with human language or for writing some template/boilerplate code. An efficient search engine rather than combing through stack overflow, or instead of reading through the docs for integrating between Snowflake and Python 😅

[@bertobertoberto3]

Which api are you using? ChatGPT?

[@machdragon]

@@bertobertoberto3 yes ChatGPT and also GPT3 for parsing earnings calls

[@Scuffy]

@@bertobertoberto3 do they have an api

[@Scuffy]

@@machdragon does chatgpt have an api

[@leonvanpersi8044]

Great Video! Would be awesome if you could make a video how to do a text analysis with python an AI.

[@mohsensafi7778]

AMAZING!!!

[@esra_erimez]

Great video!

[@prodatalab9055]

Larry, please host any reference documents in open environements.. Google will not let me access your colab and I prefer to stay away from evil oligopolies as much as I can. ;-) .. you provide great instruction videos! Thank you

[@parttimelarry]

You can copy the Python code to your local machine and run there if you prefer. I just put it on Colab for people that don't want to set up their own Python environment locally.

[@yannik9341]

I would be more impressed if you had a bot trading off the speech, which is possible, and tricky. But nice intro. I have an incident management background and I always wanted to have a bot write an hourly update based on a conference call.

[@soesevenonesix]

Great tutorial Larry! Could you explain how to get the second int object into a DataFrame without looping into the S&P data? Basically just turning the second object into a DataFrame by itself. I believe the part is at 17:45ish.

[@BA-ve7xp]

larry, when discord? so we can all digitally hangout in and hack the markets ;)

[@LeveragedFinance]

Very good comments Larry Page

[@boogiman007]

wow quickest pivot ever

[@ControlTheGuh]

This is amazing content as you can save yourself a huge ammount of time (earnings script screening). Is it possible to apply this to streaming data? Powell speech live?

[@parttimelarry]

Hey, I'll show how to do this on a live stream later this week since a few people have asked. I turned on my microphone during the November 30 speech live stream when he was more dovish and fed the audio to Whisper in chunks, and it works pretty well. If the calls don't print, you'll have to control the guh though.

[@ControlTheGuh]

@@parttimelarry hahaha. Great to hear that's possible too. Looking forward to seeing the video. Keep up the great work

[@dPointe]

This is just amazing, thanks for sharing! If I wanted a more complete TL;DR, how many would you say.. 4000?

[@deltacodealpha]

my company did this with fed minutes in 2016, this has been done

[@parttimelarry]

Sounds good. The benefit of this tutorial is that it demonstrates Whisper, which only came out at the end of 2022. Since it is open source, many people will use it for applications that you or I haven't thought of.

[@dintal-_-4121]

I like the idea you provided, will you do another with live data? Because all these calls are mostly affecting the live stock price.

[@parttimelarry]

Yes others have asked so I will share this

[@BluePanda-io]

Wow❤

[@leochan8669]

I have a small project I like to use your codes in this video. Please let me know how to credit you. Thanks!

[@edrodven]

Great job and application of speech has the limitation of understanding humans are not exactly transparent therefore speech values have to be discounted for such fact?

[@TheThreatenedSwan]

Now to correlate this with actual actions and things like IQ, moral foundations, etc, and predicting people's actions, at least in specific instances, can't be too hard, right?

[@antrikshkandhari3641]

This is amazing! Is there a way to transcribe a conversation using Open AI Whisper and also diarization it?

[@jayshay7416]

dude.. awesome. how difficult you think it would be to find a way to do this live.. think it would be advised to find a way to do as much in say, C++ in order to get the job done optimally?

[@parttimelarry]

I have tested running this on a live stream of Powell and having it place a trade based on what type of sentence is spoken. It's a bit speculative and I took it out of this video, but happy to demo it on one of the next ones if people want to see a more real time version that interacts with a broker.

[@jayshay7416]

@@parttimelarry man hopefully more people request it!. did you have to use any other specific python libraries to do it? anything you would do to make it better the next time around?

[@bakari007]

@@parttimelarry That would be awesome !! 🙂

[@negadan77]

Would be amazing to use this with ICT's content

[@arturKrzyzak]

Hi, how to enable GPU while running code on local PC? Mine is using CPU by default (I see it on task manager, also I get prompt: warnings.warn("FP16 is not supported on CPU; using FP32 instead"). I have already made some research- (os.environ['CUDA_VISIBLE_DEVICES'] = '0') but it does not make any difference. Its still running on CPU.

[@sippy_cups]

would pytube work on a live stream? it'd be cool to embed as the stream is released. then we could just periodically ping a bunch of channels of interest for going live and start listening when they do

[@parttimelarry]

Hey since I've received this comment a few times already, I will demo how you can do transcription from a live stream fed to a microphone. I have tested this already, but didn't include it since it made the video too long. Will make that part 4.

[@oshoturiya]

do you know similar project that can remove noise from audio? while preserving or improving speech

[@parttimelarry]

Are you removing noise for the purpose of transcription? Whisper can work well on noisy audio if you use one of the larger models.

[@wangjueliang]

Thanks for this amazing tutorial, Larry! How do I replace the input to a Podcast URL? Thanks.

[@parttimelarry]

With a podcast, you would skip the PyTube. You should be able to use the Python requests library to download the audio file. I may bring this up in this weekends video.

[@wangjueliang]

@@parttimelarry Hi Larry, I followed your method and uploaded a transcript of a Podcast to the ChatGPT to summarize it. The transcript is too long and the ChatGPT rejected it. Any suggestions how to work around this? Thanks.

[@wangjueliang]

This is solved by watching No.3 video. Thanks Larry. You are amazing!

[@Alexthatsnottaken]

This is so fuckin epic lmao

[@sychrov81]

Holly shit, cant believe I understand all of it 😂

[@Qushaak]

Would you share that part of code where realtime transcription is in action (how to properly break and send audio data pieces to whisper so that transcription is not far behind)? Thank you.

[@parttimelarry]

Shared on the community section

[@TreeLuvBurdpu]

Is there a reason you didn't just fetch the KZbin transcript using, I think you used PyTube. That can fetch the "Captions", which is the transcript. It's got no sentences, and poor punctuation. Is Whisper better?

[@jamestkirk3396]

Do you happen to know of an open source framework that extracts emotion from audio? The audio to speech processing loses it completely beyond usage of “sentiment words”. Especially in application to earnings announcements, where significant amount of text has been prepared, such quintessentially audio aspect might be of interest in evaluating, say, the difference in confidence between different speakers or during reading different parts of the presentation.

[@chessdad182]

When the Fed has an AI talking to us... look out. Or is one already talking to us.... oh oh...

[@hj5413]

Hi Larry, I have a big question. I just ran the code in the colab link you provided. But seems we don't have to provide the OPENAI-API-KEY, so it's free? I am confused.

[@parttimelarry]

Whisper is open source and can be run on your own device for free.

[@xs6819]

I followed you all the way to the csv file and I couldnt get the next line to accept the format 😵

[@wealthofmind145]

slay

[@sippy_cups]

also make a discord!

[@PhilipDenys]

english only?

[@varuns00]

If you're in the advanced AI training or inference space, and you're working on OpenAI APIs, you're already dead...

[@mikhail5002]

please use dark mode if you have full white screen - otherwise it's not watchable for anyone who isn't watching in broad daylight

[@RexxorDDG]

Screw Powell

[@breakitdown4346]

Too bad both the feds press announcement and stock earnings calls are already transcribed. Not really a necessary application of this, but interesting nonetheless.

[@sriramkrishna6853]

Firstttt!! ❤

[@Stopinvadingmyhardware]

All I can hear is your cop talking over my cellular network As if I were a moron. As if I cared what he thought of my medical records. Which makes me wonder how he has a copy of my medical records.

[@gridplan]

This fails for me in colab notebook at the point where it tries to run youtube_video.streams: "AttributeError: 'NoneType' object has no attribute 'span'".

[@financialsolutions3431]

I received an error at "spy['percent'] = ((spy['close'] - spy['open']) / spy['open']) * 100" how can this be fixed

[@parttimelarry]

Did you upload the csv file with price data?

[@xs6819]

@@parttimelarry I think my issue is similar, I am just learning the ropes of coding. When I saved the price data from txt to csv I believe it may not be translating correctly.

[@evyborov]

Beautiful work, thank you!