Unity Tutorial: Voice Interaction for Android and iOS

Today Matthew Hallberg shows us speech to text and text to speech in Unity for Android and iOS so you can add voice interaction to your apps or games.
PingAK9's plugin: github.com/PingAK9/Speech-And...
Stay in Touch with Third Aurora
--------------------------------------------------------
Website: thirdaurora.com/
Facebook: / third-aurora
Instagram: / thirdaurora

[@AnotherPrashanth]

Great Tutorial, I love the explanation ! I'm here because Matthew stopped uploading tutorials on his channel😥

[@gameprogramming6550]

Thank You Man ! in 2023 I was searching on Unity Store and Google for a plugin for the Android project. Working 100%. Thanks again.

[@nickwinbritto7674]

The permission window for microphone is not popping in the app so the speech to text is also not working any solutions?

[@noizu6022]

​@@nickwinbritto7674hello have you fix the problem? I can't see it too

[@aboutgamemaking]

I really thank you for the tutorial! It is GREAT!

[@ferdibayir2433]

That's amazing bro! Thank you very much ^^

[@letmedwight]

How can I compare the spoken words, e.g. like using Windows.Speech; with: "if (args.text == m_Keywords [0])", where what is said is compared with the entered sentence from the array "[0]" and the if query is then executed if it matches?

[@josephallancruz4867]

thanks for the tutorial sir. is there any way to save the audio after converting the speech to text? thanks a lot

[@WhimsyCottage]

'PBXProject.GetUnityTargetName()' is obsolete: ''This function is deprecated'. There are two targets now, call GetUnityMainTargetGuid() for app. When i change the line of code to string targetName = PBXProject.GetUnityMainTargetGuid(); it says: error CS0120: an object reference is required for the non-static field, method or property 'PBXProject.GetUnityMainTargetGuid()' Trying to get it to work on Android + Vuforia

[@shirimm]

Since a couple of months ago this plugin causes a crash on recognized speech when the pop-up is not enabled. Does not seem like a fix is on the way.

[@shilpar1121]

When used for Oculus Quest 2, Text to Speech functionality line of code --- TextToSpeech.instance.StartSpeak(message); On executing getting error as - "W/TextToSpeech: speak failed: not bound to TTS engine" Any idea how to resolve this ??

[@hehsusss]

Trying to include this in an existing project but can't get it to work. Got the Plugins (deleted androidmanifest) and SpeechAndText folder as mentioned but when I test it on my app it won't play anything. Just in case, I also build an apk from the UnityProject included and that works.

[@jzjMacwolfz]

thanks man for the explanation! really good! Encountered an unexpected glitch..ish? When the volume is up the recognition takes almost ten seconds to recognize, without volume it's instantly, I'm trying to check why is that happening

[@tombruckner2556]

Thanks for the tutorial!

[@mageshwaranr1453]

Amazing man. How to reduce the speaking speed in start speaking cube?

[@gaboandres]

Great VIdeo! Really appreciated!

[@richardmorgan9708]

Great video! Is there a better way to handle STT and TTS on iOS with Unity in 2023?

[@jt9564]

Awesome, am I able to add voice command to a button, example: start game, then the game starts?

[@robinleprince7172]

Is it possible to make it work in the Unity Editor?

[@imnotcaseoh]

Hey man, thanks for the video. I can't seem to find where the sample is that includes the text to speech and speech to text files. Could you help me find those files?

[@DaringKreatif]

hi Matthew, is this possible to speech in other language?

[@nickwinbritto7674]

The permission window for microphone is not popping in the app so the speech to text is also not working any solutions?

[@nguyenhuynhhaidang93]

thank you for tutorial, how to build with high API(ex:24), I build with arfoundation, but your plugin have API low

[@GadgetHamster123]

Was just wondering if it works on pc as well? Thinking of making a robot baddy that can dynamically say your name.

[@gilanganandaakbar708]

hello someone can help me? in my 2020 unity version, permission microphone not appear, can somebody help me fix this? thank you

[@JhollQodesh]

Hello Matthew Hallberg! Sorry for my English, I live in Brazil and I'm starting in Unity and I want to develop for IOS, I have an Ipad Pro 2020 and I know that I need a MacOS to use XCode after Unity, my doubt is: I can do the entire development process of do I play on my Windows PC and later on MacOS to use XCode or do I need to do the entire process on MacOS (install ynity on Mac too)? Another question is: I have a very low budget to buy a Mac now, if only to run XCode what is the minimum or cheapest I need to have it? (Macbook Pro, Air, Macmini)? I don't even know which old model runs Xcode. Thank you!

[@Leto]

Maybe you can try downloading MacOS to a Virtual Machine! And even rent a Virtual Machine (For example, Paperspace, that is a high quality perfomance virtual machine and you can pay for use hour). And also, sorry for my English, I'm a native spanish speaker haha

[@yasasvitennety4885]

Hi, thanks for the tutorial. But it doesn't seem to be worlong for me. The app builds and the google microphone popup works, but it does not show the recorded audio on the text component attached. This leads me to believe that it's not working properly

[@maythesciencebewithyou]

How much extra size does this give your app?

[@ashishv1185]

Hi Matthew, nice tutorial on that native textspeech plugin. Pls help, trying to find how to make the tts voice be feed to SALSA lipsync model, as the voice is playing through the native plugin. By the way, I have tried it with EASY AR SDK and text to speech working cool. Just making it work with SALSA lip sync, can you share some guidance on it.

[@user-ws1lw2kx2s]

Just saw this recently, have you solved this problem yet? grateful!

[@rutulthakkar9289]

how can i pass speech context phrases for better transcription for speech to text api from C# script ?

[@jaybob19904]

Is it possible to get this to work for a pc application I only need the text to speech function but I need it to read the text of a specific part and then speak it while it updates. Any help on this would be greatly appreciated.

[@diamondglitter205]

thank you!. Does it use Google speech services under the hood? and is it free for thousand of users? and can it be offline?

[@acayipacayipseyler8703]

Hi. This works on all android devices, but ios 14 also comes with a black screen and the application does not open. There is no problem with ios 13.

[@farzanaamin5717]

Hi, is it possible to implement the speech for android in MAXST AR?

[@jjaylee6095]

Why do I click two out, one down and one up, and then when importing to the phone, press the blue square and it will flash back

[@dnslv_]

Hey, man. Do you think it's possible to achieve continuous speech recognition with this plugin? At the moment I coded my application to call StartRecording() as soon as it presents an error callback. But this is not very ideal. Thanks for the tutorial!

[@Logan_Wolf_Online]

(keep in mind, not tried this on mobile, so milage may vary) You could probably run it through the microphone command parsing loop you are using, have a command_level variable that can switch between command_level=0 (for detecting listen command), command_level=1 (for basic commands) and what not. have the "Listening" method run on a loop from the start, but have it start with command_level=0. Use whatever string query system you are making use of within a switch(command_level) block. It should not be too difficult to port that over to mobile (again, not tried mobile voice commands yet myself).

[@aayushjain9562]

Great tutorial. Any resources or leads to use the generated audio from text to speech for lip syncing?

[@alexbethke8665]

Microsoft Azure can provide text to speech voice generation along with visemes data for lipsyncing

[@soundmartell]

Hey, thanks. When I copy the plugin and SpeechAndText folders to the asset folder i get this errors: 1- 1 - The type or namespace name 'iOS' does not exist in the namespace 'UnityEditor' 2 - The type or namespace name 'PBXProject' could not be found (are you missing a using directive or an assembly reference?

[@omaral-laham8655]

make sure your version has ios support available

[@muntasirangeles7722]

how to fix the crash when it detects what you said? please help me coz this for my capstone project :(

[@mkousar11]

Can I run it on Nreal Glasses Computing Unit as .apk file?

[@leoportopassos]

Can someone help me? How do I display a message (eg, "I didn't hear what you said") when the user doesn't say anything and so the system doesn't hear a word?

[@bardockevans6014]

Hello good afternoon, I have the following problem: when the start listening is executed in the mobile build, it crashes. I'm not sure why this could be happening. Any ideas?

[@robba-tv]

Greate tutorial. but not working on recent android version device with STT. I Couldn't find the reason :(

[@newstrangerX]

how i can remove the sound of start listening is started and end sound? please

[@JoaoPaulo-mk8ro]

I subscribed a method to the OnDoneCallback but it's never called, does anyone know how to solve this?

[@to_ciex8760]

how to add/change to local language? and how to check available language? thanks

[@kasundodanwala]

I get this error saying "probably the android SDK is read only", which doesn't seem to be the case. Something is wrong with the plugins or assets. There're no compilation errors. It's just build errors. Anybody know to fix them?

[@jocelynfrancis3067]

is it possible to use this code in PC? (MacOs specifically)

[@Glitchesftw]

For Android, when I click the microphone button in the Scene I get ERROR: Insufficient Permissions. Then when I let go, I get ERROR: Client Side Error. I have already allowed the Microphone Permissions. Can anyone help? SOLVED: I had to re-enable Voice in my Google App permissions

[@javokhirbozorov2120]

Bro how did you add ui text to the voice controller.

[@user-lw2jc2qf3i]

Hi thanks for your tutorial. May I know is there any way to use a UI Button or Image as button instead of 3D cube? Thanks in advance!

[@gilanganandaakbar708]

hello, r u know how to make it now? im stuck in this problem, can u help me how to use button or image instead 3d cube? thans

[@generichuman_]

Little tip, the thing you are trying to do has nothing to do with speech to text, it's plain vanilla Unity functionality, so instead of asking the creator of a very specific Unity video how to do something basic and hope that he has time to get back to you, just google or KZbin "How to create a UI button in Unity". You'll get a million results and you won't have to wait. Learning how to find the information you need quickly is your biggest asset as a developer. 99% of my job as a developer is googling shit.

[@levrisfirst3291]

It ONLY workeble for android and no WINDOWS support ?

[@rnjt8960]

Hello admin, now I start learning develops android apps, I done every steps that you show on this video, everything look fine until I clicked record voice on my phone, the program has crashed, it seems there is only voice recording that leading to this. After it showed what I speak then it crashed I don’t know how to fix this 😢😢😢

[@hajarakotoarimahaisaiaprec8452]

First of all big thanks for the tuto. I have a question. For the TTS French language is available?

[@nickwinbritto7674]

The permission window for microphone is not popping in the app so the speech to text is also not working any solutions?

[@globalhellinside2967]

i try install speech.apk from plugin but SpeechToText doesn't work

[@christianbellamy8726]

Could this be used on the quest?

[@jingf5356]

Amazing tutorial. The TTS voice on my Android device sounds really weird, nothing like yours. Is there any other language code I can use besides "en-US"?

[@jingf5356]

I found the reason. It's the TTS voice lib on my phone.

[@nelsongomez8547]

Hello friend, I hope you're very well. Excellent your video. Is possible use ML-agents in this project? If you answer is "yes", Could give me some resource, video... As help, PLEASE.. I need information for i begin

[@perpixlstudios5144]

HELP PLEASEEEEEEEEEE................................ How can I compare the spoken words, e.g. like using Windows.Speech; with: "if (args.text == m_Keywords [0])", where what is said is compared with the entered sentence from the array "[0]" and the if query is then executed if it matches? THANKS IN ADVANCE

[@rejepdurdymeredow5893]

HI bro.Did you solved problem?Can you give advice

[@tiger.gamedeveloper6582]

can this be used in vr too?

[@mathieutimba8996]

Hi , thanks for this tutorials. I got a XCode build error : Undefined symbols for architecture arm64: "_OBJC_CLASS_$_SFSpeechAudioBufferRecognitionRequest", referenced from: objc-class-ref in SpeechRecorderViewController.o "_OBJC_CLASS_$_SFSpeechRecognizer", referenced from: objc-class-ref in SpeechRecorderViewController.o ld: symbol(s) not found for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation) any idea ? Thanks Mat.

[@mohamedmmc]

same i got the exact same error, did u handle this error ? please tell me how

[@Quantum_Tripathi]

Add speach recognition platform in your xcode your problem will be solve

[@vineetbhat1114]

Hi, I did the exact thing as in the video but my android app is not asking for microphone permissions. Can anyone help?

[@tborth]

Same issue ?? Did it fix

[@davoodjoon]

Hi, how record voice in android smartphone ( unity3d )

[@THEJABTHEJAB]

This works perfectly on iOS when launched using Xcode debugger. But will show a blank screen and crash after a few seconds if run directly on the device. Tried Unity 2019 and 2020 and two iOS devices. Any ideas?

[@DimitryArsenev]

On android its crash too. I think its make crash on many systems.

[@rnjt8960]

My Android too, any idea how to fix

[@tborth]

I my phone it not ask for microphone permission

[@Quantum_Tripathi]

This work fine but i need audioclip so that i can perform lipsync. Thanks

[@toannguyenxuan4291]

Thank you very much!!!

[@Miablosumimasen]

Is there anyways to have this working for phrases? I can get the audio to be picked up but not sure how to have phrases impact gameplay

[@amuzak9063]

I'm working on something similar, you could have it simply look for certain keywords in the string that stores the voice input, then fire a corresponding callback based on the word/words detected. It won't be very modular, or very extensible right off the bat, but it's a starting point

[@Miablosumimasen]

@@amuzak9063 Did you just use an if statement to have the code look directly for that string? Honestly, I'm not super strong in coding but I did manage to get it to work pretty seamlessly using Unity's Native Voice recognition stuff and on PC but still struggling to get it to work on mobile.

[@RitikAgarwal21]

will this work with oculus?

[@user-vf7be9nx3i]

does any one has the problem with textmeshpro not updating?

[@ramkeshsingh4051]

Can we use in oculus quest 2?

[@itayhilel2168]

loved it

[@KingJames586]

Not working at all at the moment. Assume the repo needs to be updated which will require a Java rewrite and recompile.

[@2-TheMoon]

how to make using mp3 file?

[@mrji3g4xjp6]

You are my HERO

[@audacityfilmsproductions5334]

How in the world do you get this to work for iOS??????

[@kora_lee]

9:40 Thank you for the tutorial!

[@dclxviclan]

Perfect practice

[@beFrah]

" 'TextToSpeech' does not contain a definition for 'onDoneCallBack' and no accessible extension method 'onDoneCallBack' accepting a first argument of type 'TextToSpeech' could be found" HELP PLS!!!

[@nitroidalrashid2157]

Hey its not 'onDoneCallBack' with "B" to Back but back so its should be 'onDoneCallback'

[@hashanmadusanka6128]

Thank you so much

[@alfonsoibi]

Hello, I have a problem when I run the sample application on a samsung s22+, when I press, it starts to recognize the text perfectly, but, when I release, the application crashes. Any idea what can happen? thanks and great job!

[@generichuman_]

Same issue, let me know if you figure it out

[@generichuman_]

a temporary workaround is to re enable the "is show popup android" bool on the speech to text script, then just use the popup modal. This resolves the error. Not perfect though, still want to figure out what's causing it. I noticed in the code that when the bool is false, it calls a java class from com.starseed.speechtotext.Bridge, so maybe it's a java issue

[@jessycafonsecaaraujo385]

​@@generichuman_ Hello!! I had the same issue and thanks to your comment, it worked. Have you found out how to make it work without enabling the popup? Thanks

[@hassanbennajma5155]

how to make it work without enabling the popup?

[@tonytran07]

Wait...... Android and IOS has STT and TTS for free built-in?!!!!

[@ashlington79]

Build gradle error on android due to deprication, will not build apk on latest version

[@ashlington79]

You have to remove 'android:minSdkVersion="16" ' from the manifest.xml to fix this

[@Pavetl]

I think this plugin is really buggy. Anyone has an update to make it work? I would appreciate it

[@airsupply1606]

Hi admin, how to do if I want to change scene using voice command?

[@themeatynoob3886]

Use "using UnityEngine.SceneManagement;" then on voice command do "SceneManagement.LoadScene(*SceneName*);* That should work

[@airsupply1606]

@@themeatynoob3886 thankz admin 😊👍🏻

[@criadabatterista8647]

Hey this is a freat tutorial, but when I tried to do speech to text, it starts pop up a google interface, and not changing my text :(

[@criadabatterista8647]

nvm, he does explained it, I wasnt really listening XD

[@ivanrodriguezovin]

Hi everyone! First of all, thanks for the tutorial! TTS is working fine, but STT isn´t. What could it be? My OS is Android 11 and the LANG_CODE i´m using is 'es-ES', i don´t know if that affects

[@aryanjumani4524]

same

[@aryanjumani4524]

for me, it opens up google's tts and then shuts it

[@LucasADOrnellasSPinto]

same

[@logan56]

Same : (

[@hafizahmad3501]

same 🥲

[@melodyuool8771]

Hello. This tutorial helped me a lot. but STT isn't running. I think StartRecording works, but the application is forcibly closed for some reason when the StopRecording. I want to analyze this, but i can't see the plugin. Do you have any solution? My phone OS is Android 11.

[@atharsiddique7976]

@A Editor Any solution?

[@atharsiddique7976]

@A Editor Same issue i am facing

[@hassanbennajma5155]

Same issue i am facing

[@rnjt8960]

Any solution😢😢😢

[@AvivXR]

Great tutorial. Very helpful. Is there a way to start listening to speech without pressing a button?

[@audacityfilmsproductions5334]

That’s what I want to know, I switched from Windows to iOS and everything’s wonky.

[@klortak]

did you get this figured out? I've been trying to but I'm pretty new to coding in c#.

[@vedantkadam2147]

@@SilverEchoDreamer ya i did the same it works

[@surferlab2829]

Hi @@vedantkadam2147, Could you share how did you do it please ?

[@resulmustafa4070]

Hello I couldn't do without STT android popup. Popup does not work when false. Please help me

[@dubeybhai6918]

thanks

[@masterchange8155]

Great tutorial but when i come to run STT the app crashes. I am developing on Unity 2019 and running on a Pixel 6A. I have triple checked the code and it is the same as yours... so any idea how i debug? Nothing appears in console before it crashes... TTS works great thanks for any help you can offer

[@thevisualshaman]

Have you been able to find a solution? I am coming across the same error

[@thevisualshaman]

@masterChange8155

[@hassanbennajma5155]

Have you been able to find a solution? I am coming across the same error

[@masterchange8155]

@@hassanbennajma5155 no - i ended up buying a solution as wasn't getting anywhere on this - one small issue flummoxed the whole thing

[@snubnosegenopeak]

Great tutorial! Thanks! Anyone know how to get rid of the beep notifications sounds?

[@ashlington79]

Hi, did you ever work this out? its very annoying :D

[@hasancolak2270]

For now, the only way I know is to set my phone on silent.

[@justinwhite2725]

Likely not possible. It's a security/privacy feature of the phone (same as the shutter snapping sound whenever you take a picture with your phone)

[@ashlington79]

Got this working now, its great :) I would like to know how to mute the beep notification sound (uses ringer volume on Android) when it starts & stops recording. Thanks

[@MagicalSAMElectronics]

i want to know the same...Did you find a solution for this?

[@ashlington79]

@@MagicalSAMElectronics no unfortunately, I suspect it's something that would need to be done within the plug-in

[@MagicalSAMElectronics]

@@ashlington79cool...thanks

[@leoportopassos]

I have the same doubt... Can anyone help, please?

[@mohammedshalu5525]

it's worked fine under IOS 15... Not working in IOS 15 Anyone solved this..?

[@aboodriazy3387]

what about webgl

[@pointpic]

Is it still valid in 2023 ?

[@miqdadnaduthodi7193]

Got an error SpeechToText does not contain a defintion for instance. Pls help

[@kaanvardar4575]

make it Instance, not instance with the letter case sensitive

[@DimitryArsenev]

Plugin not work anymore - github project just make crash app.

[@ravinduyasith9548]

Text to speech is working on my android, but when i press the button for speech to text the application crashes..any fix for that?

[@Need2pee]

Hey which phone are you using please respond? For me it doesn't work at all It shows the pop up for the voice mic but does not update the text box

[@ravinduyasith9548]

@@Need2pee Google pixel 3.

[@mariacamilacalvosalazar4256]

does this work in Spanish?

[@jos_valentin5810]

Does this plugin includes support for spanish or other languages speech?

[@softhoarders12560]

yes, it does

[@geek_dev]

please source code 😥 url