7. OpenAI Whisper and GPT-3 - Voice Commands and Live Transcription

[@parttimelarry]

Like the video? Support my content by checking out Interactive Brokers using the link below: www.interactivebrokers.com/mkt/?src=ptlPY1&url=%2Fen%2Findex.php%3Ff%3D1338 Source Code: github.com/hackingthemarkets/openai-whisper-voice-commands Note: Although this made for a workable demo, there will be a number of edge cases to handle. I found that I learned the most while trying to handle these edge cases. I also probably should have used aiofiles to read the transcript.

[@shrvn110]

I have only watched 49 seconds of this and I love it and I love you.

[@k00a7]

Keep up the good work Larry. I always feel happy once I see your video.

[@rajum9474]

Wow !!! what a great tool. I am super excited to learn and try this out.

[@bakari007]

Thank you for the video !! Excellent!! Very interesting ! It opens the door to many possible ideas! The next step can be an order opening with a condition? For example a simple voice (prompt) message may refer to an analysis based on several statistical indicators, if the conditions are verified at x% then open the order (multi-layered condition reserves!)! That would be a great time saver! Thank's Larry !

[@parttimelarry]

Thanks, I don't think that voice commands necessarily save time (at least not at this stage of the tech). I just think it is cool how you can chain these tools together to do interesting things, it's just kind of "neat" to me. The same way that I find Alexa voice commands kind of cool, but don't use them all that much in practice. Perhaps as the tech evolves in the coming years, it will get smarter and faster to the point where it does save time and is more enjoyable to use than pressing buttons (like the movie Her). The ability to do live transcription is very useful on its own though and definitely saves energy for these kinds of tasks.

[@stephennn403]

​@@parttimelarry Hey Larry how can I add this plugin into Brower to correct my English writing?

[@transparentinvest]

Awesomeness!!!

[@nathan3480]

I tried whisper today for my chatbot and when I got quiet it would transcribe the silence to "Thank you for watching" in japanese so im glad to see it isnt just me going crazy

[@shrvn110]

Thank you brother.

[@eltoroloco28]

These videos are amazing! Thank you for putting so much effort in to publishing them all

[@FRANKWHITE1996]

Thanks for sharing ❤

[@JeffAporta]

Gracias, me sirvió mucho

[@fromtaiwan100]

Really interesting! Thanks!

[@MachineAI]

Thanks for sharing these videos. You are the best on youtube. A question, in general, is not specific to your videos. For data security reasons, How can I Trust Python Libraries that I am importing? How can I find out what is happening behind the scene?

[@robertmccully2792]

Your a genius.

[@DavidDji_1989]

6:40 : I think it is actually ghosts trying to talk to you 😂

[@Zynthh]

Awesome stuff! Saw you made a gridbot with ccxt in python earlier, any chance you could show us to how to make a gridbot with freqtrade?

[@luisgarciarios4988]

Amazing, this is something that I waited long time ago, is it a way tu run that in google colabs?

[@wordpressobsessed9067]

Very cool, excellent presentation! Is it possible to modify this type of app to create a real-time translation application? For example, two people log into a webpage and speak different languages like English and Spanish. The english person's speech is translated to Spanish and written on the screen, and the Spanish is translated and written in English on the screen. Is that possible?

[@wigleboy]

Sounds dangerous for me -- "No I meant 10" Always nice work though; thanks.

[@haminton46]

Can you make some content to summarize an arbitrary video to a 30s video? Is this possible with current openAI?

[@loafandjug321]

Whisper, profit one million dollars today, next please.

[@MachineAI]

I am trying to get 1 min of intraday data for stocks and options, but I do not see an easy way around it. I tried different data providers, but the data does not sound good to me looks like lots of missing data.. and that is crucial for doing any backtesting. Can you share your expertise on how and where I can get stock market data?

[@parttimelarry]

I made a tutorial on Interactive Brokers for this and also included some code for second bars in the Whisper video

[@hanael-sherbiny6709]

Can this be used to send commands to a code editor like vs code for example? So the speech or audio would include something like "command v" for example and then that would be transctibed then somehow it would be read and the command would be executed

[@wesremy]

Question, will putting the audio in chunk not have the risk for ending the chunk in the middle of a word? Causing whidper to not catch the word

[@saumitrakarmakar8336]

Can u make a video on how to backtest crypto options strategies on python looked everywhere couldn't find a answere

[@pritambagal2679]

Could you please do binance webhook for trading view again? As heroku is no longer supporting for a free plan. It would be helpful. With advancements.

[@pypypy4228]

Do you code this presuming ChatGPT uses davinci-003 model? ---- Are you built on text-davinci-003? No, I am not built on text-davinci-003. I am built on a version of GPT (Generative Pre-trained Transformer) which is a type of language model developed by OpenAI

[@pypypy4228]

Sorry, my fault. You never talked about chat GPT in this vid

[@pypypy4228]

Awesome vid btw 😎

[@casadomago2021]

I can do anything similar with android?

[@MikeBtraveling]

Dude, feed that test response back into OpenAI, transcribe to narritive audio and play it using Siri's voice and youve created the Star Trek Computer UI...

[@bseddonmusic1]

I think there may be a better way to do this though it will mean stepping away from Python and Whisper for the speech to text. If using Python and/or Whisper is a critical feature then this is not for you. All webkit browsers (Chromium, Edge, Safari) include speech to text and its pretty good. This means its easy to use the browser speech API to collect text using JavaScript. Because JavaScript can execute asynchronously it's possible to determine when the user has completed their sentence - if there is no new text appearing in the buffer for, say, 5 seconds, assume the command is complete. I think this is better than chopping up the audio to see if some of it is soundless for a long enough period. A simple REST endpoint on the web server (the same one which serves up the page that initiates the JavaScript to implement speech to text) will allow the JavaScript to be able to send the text of the command to the server for processing. The server, presumably implemented using Python as you have done in earlier videos, will then process the text using GPT-3 in the way you describe. It can send a reply back to the browser to let the user know if the trade was successful or if the command was not understood. After your video on prompt engineering I implemented a prototype of allowing a user to verbally describe the layout of a pivot table using the technique above. The text is captured using the browser speech API and sent to the server and then on to GPT-3. GPT-3 is used to see if it's possible to extract information from the text about the dimensions to be presented and the axes on which they should appear. So far, the technology works reasonably well. The problem is the human as people seem to find it hard to describe in speech the structure of a pivot table. Instead it's been necessary to have the user describe 'blocks'. For example "show me a balance sheet for 2021 and 2022".

[@parttimelarry]

Cool! I didn't know about browser speech to text, thanks for letting me know about this. One note on what I did here though - I am not waiting for soundless periods, I am using the word "Whisper" in this case to indicate a command is starting, similar to how you say "Alexa" or "Ok Google".