Using Cameras to Hear Better in Noise

How Google Ai could help you hear Better in Noise with hearing aids using cameras. Dr. Cliff Olson, Audiologist and founder of Applied Hearing Solutions in Phoenix Arizona discusses technology that already exists to help your hearing aids perform better in background noise using cameras.
Google Ai: research.googl...
hearingup.com
drcliffolson.com
appliedhearing...

[@glengosling5636]

Bring it on ASAP.

[@jorgequiroz881]

research COCOHA hearing aids, that is the future

[@kennethdjordan]

Thanks! This needs to go viral. Just imagine the social implications of this improved communication capability and accuracy that will bring all the newly empowered hearing-challenged people back into the fold, no longer isolated and hesitant to put themselves in difficult auditory situations.

[@peterdelbove2838]

@@kennethdjordan so true!

[@WWPlaysHoldem]

My biggest gripe about my hearing aids. That would be a great improvement.

[@tedlassagne8785]

Cliff, this is an amazing development. Thanks for bringing this to our attention. The power requirements may turn out to be less of a problem than you anticipate.

[@NightBazaar]

Pretty remarkable. It would also be important to NOT mask out important sounds like sirens, warnings, etc.

[@DrCliffAuD]

I'm sure the Deep Neural Network could be trained to keep those sounds. However, I do agree that some awareness of other sounds in your environment would be important to keep.

[@jackielulu1676]

Not every environment would be appropriate for this technology; like while driving a car! You'd need to be able to easily turn it off and on. I wonder how much of the AI could be offloaded to your smart phone.

[@RossAMcKay]

Exciting development to help deal with the common limitations of current hearing aids.

[@c016smith52]

I think this would be great in tandem with Meta Rayban glasses (assuming those are mainstream in time, they seem quite affordable and practical if they're not socially shunned like Google Glass) and can be used in 'hearing aid' mode that allows cameras to identify primary speakers and then hand-off those audiostreams to your 'paired' hearing aid. Love it! Edit: just got to your reveal on Meta, totally agree. :)

[@jackielulu1676]

I think every hearing aid wearer would love this!

[@Graygeezer]

I already can barely afford hearing aids from my audiologist. Can you imagine the cost of this miracle technology?

[@kimstockdale632]

like many new technologies (think cars, lasik vision correction, computers), the wealthy will purchase the initial units. But then as technology improves there is high financial pressure to drive the cost down so smaller unit profits but higher volume results in higher profits for the company. And the end user ultimately benefits. Thank you free market capitalism.

[@rafiquekeshavjee8033]

This is really important. Whoever succeeds will make huge amounts of money.

[@anthonywatts2033]

Explaining anything simply and briefly is a concept "experts" from the US cannot grasp.

[@robertfletcher]

This sounds amazing. I'm at the early stage of needing a hearing aid. I won't be able to aford something like this until it's proven well enough to be covered by insurance. I'm a big technology freak, so I'm into any increase in technology.

[@independent5564]

That would be wonderful, I hope the VA can get these.

[@harisree668]

Please kindly do a video on hidden hearing loss accompanied with mild hearing loss and best hearing aids to improve this type of hearing

[@charliejg]

It's basically video editing software being used to assist with hearing by separating the various tracks. But, it has to happen very quickly to be natural. Besides the processing and battery issue would be the speed of the wireless connection between the camera and hearing aid. But, you could have a wired connection from the arm of the glasses just like behind the ear devices. As you said, it's a matter of time.

[@MrTennc]

This ability to improve hearing in noisy environments would also be desirable for people with normal hearing allowing a huge market. I worked in a noisy environment for years and incorporating this ability into hearing and eye protection devices would be easier than hearing aids since the safety devices are so much larger.

[@nbaylor1]

Would these aids separate background noise (ie music) from actors speech in tv shows?

[@kenpeterson3414]

Sounds great, but I can only wonder what the units might cost..?

[@rpgiacon]

I guess that this will hit the spot when theys combine this with microphone arrays. Microphone arrays can do the same much faster but the hardwares are not fully developed yet. I bet in 2 years we will hear about some mems of micro arrays that will make it portable to use as a hearing aid device in 4 years

[@MrTimelinegb]

I would like a microphone camera connected to my glasses so when I look at a person it would automatically do what you showed on your video into my only good ear. If you could also produce noise reduction and I have a control for my iPhone that allows me to just the background noise down to minimal this would be outstanding. So the ear peace/ hearing aid that would go along with this system and the camera would all have to be Bluetooth. It will also provide me with Voice hearing for the opposite side of my good ear as I only have one good ear. My name is Gary Brandt before I lost my hearing in one ear I was an audio engineer for 55 years.

[@benqurayza7872]

They need custom chips to get around the costs and energy requirements and to squeeze in the AI capabilities.

[@BernadetteSwetz]

This would be a tremendous step for me. Background noise is a big problem.

[@macrumpton]

Imagine how much better this tech can be today with the AI developments in the last couple of years! As far as training models we have quite a bit of video of people talking available. The idea of trying. to do all the processing and have the battery capacity crammed into an earpiece or even glasses is just crazy. If only we had a powerful computer available everywhere we go, that had battery capacity to run all day! Like your phone! The earpiece and cameras should not have any processors on them, just batteries and radios to communicate with your phone where all the processing is done. A big advantage is that most modern phones will have powerful AI optimized chips in them already, (far better than you could cram into a earpiece) and all that is needed is an app that can make use of them. As an app it will be easy to update to improve the user experience , and most people get a new phone every few years anyway.

[@ruthdallas6125]

I agree background noise is a huge problem but I think there is a lot more work needed to have hearing aids live up to their hype. I have normal mild hearing loss of high frequency yet my Rexton hearing aids have not helped me hear my women friends when they don't speak up, or are not looking right at me or when there is even mild background noise. I had a audiologist hearing test and the tech adjusted my aids to my audiogram twice, she made lots of other adjstments to help me hear my friends but I just hear motors, fans etc really well now I notice NO IMPROVMENT AT ALL in hearing women's voices. I've decided after reading and watching a lot about this that even though this is the major problem for a majority of uses the aids just aren't very good at doing this one improtant thing at all. Thanks for all your good information.

[@DrCliffAuD]

I always have to ask what your Speech in Noise score is, and if your hearing aids are programmed to match your Hearing Loss Prescription as verified by Real Ear Measurement?

[@lewislaw6835]

Your research is spot on.

[@fabianr.straube751]

Hey Cliff. On point. Was just talking about this with a fellow AUD last week. Why do you say 6 years at the end? Anything you know?;)

[@tedmacfadyen5702]

It would be awesome to be able to eliminate background noise. It will likely be a few years away and the cost will be prohibitive for a few years. Being in my mid 70's it will probably be for a much younger generation of people who will have hearing loss. But it is exciting news

[@GdncHfjbdkf]

Its a eye glasses made with infrared sensors that connect to your brains stem via your cranial nerves! Medial and Lateral Geniculate Nerves either through an infrared sensor chip inplabt or external wearable

[@billrogers1008]

Wow! I am very interested in this. I'm sure it will help me.

[@Thomas-pq4ys]

This won't work for me. I want the big picture to be clear. Sometimes the person I want to hear is not in front of me. What about music? What about sound quality? I'm a musician. I have listening skills, and an audio pallette that needs to have nuances that affect sound quality. Am I screwed?

[@vhe11]

So why isn't it on the market yet??

[@GdncHfjbdkf]

Can we sample a video upload to edit video?

[@stericksonyt]

Very encouraging-and extremely important to hearing aid users.

[@colinhartridge6326]

I assume this technology will eventually be available for Phonak hearing aids?

[@mtoinma]

why do you need that brand in particular?

[@kennethdjordan]

All we need now is two well funded, effective, visionary entrepreneurs each racing to be the first one to bring this to market in an affordable form. The glasses design would be the key to available space for the battery, and the glass case would be the charger. 6 years?? My imagined visionary says 3 years. It would be an explosive opportunity for eyeglass designers/manufacturers as well. This is a captive market!!

[@pagzplay]

So yes bring it now. Hearing Aids I have get before have more Amplification possible and Bluetooth Hearing Aids now but Glasses help with understanding Speech and my other Hearing Aids did not do what Glasses do now so I like lot.

[@wintermoonomen]

Interesting! But is it just me? The AI seems to make the focal voice sound 'tinny'

[@StefanBleeck]

the problem is and will be latency. All the computation has to be done within around 10 milli seconds. That is a fundamental issue that is orders of magnitude's outside current capability,

[@tzz615]

This is amazing.

[@martinraboy5971]

Definitely want it..

[@barryalanlevine]

glasses should be able to contain larger batteries for added processing. And potentially additional mics for more triangulation of sounds. The future looks bright. And more stylish.

[@windsurfertx1]

Bring it on

[@karlaatherton]

Very clever, amazing!

[@sincerelyyours7538]

Interesting approach, but in a sign of the times it seems to me that cameras that can lip-read are not likely to work well if we're continually being urged to cover our faces with masks to prevent the spread of disease. They would only work at home or in restaurants and other places where people are allowed to take off their masks.

[@amyooi884]

Hello dr ❤

[@Gabrieleengstrom]

But what if you need glasses

[@SuperStewfart]

As a 77 year old in the prime of my life, I have spent A$11,000 on hearing aids that help but are not really effective. I am happy to spend what it takes to improve my hearing result. Background noise is the killer. I suggest that the fast chips are available from Nvidia and the cameras are available. Nvidia has driven down the energy requirements of their fast chips and batteries may not pose a problem. I am very interested and engaging in AI as I can see the benefits.

[@RobertSutkowski-l7h]

Magic 😊👌👌👌👌👌

[@jeffreystewart6571]

WOW, but 6 years from now may be too late for me. Thank you

[@dave_dennis]

I don’t see the processing being able to happen on the body. Not in the next 20 years. More likely there will be a 5G connection to a processor in the cloud. I would not mind at all carrying an addition iPhone size piece of hardware if it enabled this capability which would be game changing. 😮

[@GRILL332]

Wait for people being attacked because they wore their camera glasses into the public bathroom. Remember google glass?

[@SailorGreenTea]

Interesting

[@ksart100]

​@@SailorGreenTea how about non-reporting glasses for these purposes

[@kellismith7908]

Cool

[@MrArdytube]

I noticed worse hearing with people wearing masks during covid

[@shulamussnug7120]

Sounds like more money than average people have.

[@DrCliffAuD]

Just depends on if the market is large enough to drive the price down.

[@gastube22]

Maybe glasses could accommodate larger batteries than a behind-the-ear hearing aid, to supply the extra power needed. But the chips would still need to be super-efficient on power.

[@peterdelbove2838]

This is a phenomenal accomplishment. If this concept could be developed into hearing aids.....GAME CHANGER!

[@esecallum]

used for spying mostly

[@vinaytopa5534]

It's fantastic for people like me who have a severe problem of hearing in a noisy environment, specially in a closed space due to the echo.

[@davidmccarthy6061]

My Phonak RIC sits right on top of my ear so I can see them adding a small camera module facing forward, although I don't have any hair in the way. Glasses are the way to go so yes, some "reverse streaming" for it to use my phone processor could work. For the people that will be able to afford 10 grand for hearing aids.

[@MinciHH]

Very interesting video, well researched!

[@ricjona1069]

That (sight and sound) combined with motion sensors (three dimensional space) is the technology trifecta.

[@corvinyt]

As a deafblind person, this would be AWESOME!!

[@charlesheyne9081]

This is fascinating technology and I hope it comes to fruition. You hit all the big points in terms of all the issues that need to be resolved power, camera etc. but I wonder what the price tag will be for this incredible technology.😳

[@KYFHOme]

Imagine adding a language translator to this.

[@SailorGreenTea]

Interesting

[@lauramaeda7214]

I know that visual cues are very important in social interactions. Even though i do not consider myself a speech reader, i benefit greatly from being able to see the speaker’s face in a conversation. I even mention that if i can’t see your face, i can’t hear you to get the additional visual cues if i am struggling in a conversation. I know it’s a bit of an exaggeration that i need visual cues but is so less stressful in social interactions. I already wear glasses so adding a hearing feature to them is not an issue. Hope this technology will be available in the future.

[@pagzplay]

I read lips and I use facial expression to help understand Speech. If Hearing Aids can pick up some of it and you have cues it help lot understand Speech for me.

[@esecallum]

i am poe faced. chines are inscrutable

[@WillosRock]

What's interesting is some time ago I started focusing on people's lips when talking to help me hear better. So, anyone can improve their hearing just by doing this. Like any neural network the brain will improve the more you do this. Just look at the person's lips when they are talking. As for this technology amazing. Getting AI to do lip reading and incorporating it in to audio processing is a great idea.

[@alsjogren7890]

My hearing aids help in a noisy environment - mainly due to directional focus. Outside of noisy environments, I do not tend to use my hearing aids. When I added remote microphone either with iPhone or with the hearing aid manufacturer's remote microphone (Android had too much delay causing almost an echo), the problem got much, much better. I look forward to the merging of audio and visual to cleanup the audio.

[@stevefortney9329]

Amazing! When can I get this technology?

[@mtoinma]

yeah, my first thought was "will I even still be alive?". haha.

[@gbkworf]

That is absolutely amazing. Can you say Holy Grail of hearing in background!

[@johnm7249]

Training for multiple languages .... Is ubiquitous wifi required?

[@Jewelzoftheisle]

Very interesting...this technology would be amazing in imrproving interaction and connection one on one...

[@Robin-sy1ly]

As a person who needs hearing aids and greatly struggles in noisy environments....this is awesome...I want this technology!!

[@BobJones-r6o]

Background noise is definitely the worst problem for me.

[@scottgilsdorf2938]

I noticed the benefit of watching ppls face in my experience. Im excited aboutthese possibilities! Thanks!

[@henrybrensing3284]

I’m interested send me information . This could really help me to hear better.

[@pagzplay]

How much battery power would you need to have the Glasses and Ultra power Hearing Aids? I have lot Amplification so my Hearing Aids use more Battery power so they would need to have different stronger Batteries if I had Glasses that gave me Cues too but I know that it would help lot for me.

[@harisree668]

Dr Please help, Already purchased phonak virto p 30 cic hearing aid due to mild hearing loss by watching your videos but no cure ,now my audiologist says I have hidden hearing loss after performing some tests,mild loss in normal frequency as well as in ultra high frequency can you please suggest which type of hearing aid do produce better results for hidden hearing loss Me leaving in india difficulty for reference age 30 , difficulty understanding speech

[@ryantwombly720]

I think the battery concerns are exaggerated. Each component has its own battery - glasses, hearing aids - and if much more processing power is needed than currently available, there’s room on the glasses for extra cells. Six years is time enough for a third or fourth generation device. Get that 1st Gen out now, Google! I know you read these comments.

[@dennis5188]

Hearing in noisy spaces is a nightmare. I use Oticon, and it is good, very good, but it still lets me down in noisy environments, so I'd love something like what you just demonstrated in the video.

[@thomjohnson1292]

Wow, I need this.

[@LarryRobinsonintothefog]

Helpful, but how about warning sounds but I guess you could react to others.

[@cliffordmjordan]

I think we are closer to having a working solution than you think. The processing power when executing a trained model is really low. It could easily be done with current smartphones. Only the processing power required to train the models needs to be magnamonous (eg. in a data center). Raybans with cameras that have smart photochromatic lenses that allow me to wear the glasses indoors and outdoors with this tech would work. All that is left is an app to execute the model on the smartphone.

[@harrygmarshall5157]

This is unnatural, and seems to be from the Prince of the power of the air . I am nearly deaf and will choose to remain so , before bowing down to a I 🙏🏻🇺🇸🇳🇮

[@72Calbear]

I definitely want to technology in my hearing aids! I have so much trouble when as a teacher I’m in a classroom with several students and I can’t even hear the person in front of me 5 feet away of the background noise.

[@lesbendo6363]

Wow! What a great solution. You would have to find the "Tesla" of the hearing aid industry! 🇨🇦

[@ajaynahar9373]

Is it available for purchase?

[@tomreingold4024]

I would love this technology. It's a sensible design, too.

[@donmarsters104]

Would love this in my hearing aids. My head hurts when going out to busy places and trying to focus on only the person speaking.

[@oidbio2565]

It’s already happened with Apple AI and the AI is on the M4 chip locally so there is no need for delays due to upload and download speeds.

[@andrewhutchison8701]

This would greatly solve problems for my hearing because it is so bad. I even thought of going to a lip reading school

[@alicereed1894]

That would be awesome, thanks for sharing that info!

[@snark907]

I don’t understand how this video is supposed to be helpful.

[@allinad]

That would be an amazing solution

[@drewwilson1477]

Absolutely it will come. Curious why you said 6 years as opposed to 5 or 10?

[@GM-nr2jq]

Wow! I would love to be able to have that kind of technology!

[@terriannolson8923]

I think this speech separation is very cool 😎

[@larrylawson5172]

Very interesting. When can I sign up???

[@dougleppard]

This would be great

[@samchavez8994]

this is interesting but how will it be used with hearing aids?

[@Malakawaka]

Spy technology for the people.