How to Build a Multilingual AI Voice Assistant in FlutterFlow (OpenAI Text-To-Speech App Tutorial)

In this video we walk through how to build a multilingual AI voice assistant app in under 35 minutes with FlutterFlow.
Cloneable Project Link
app.flutterflow.io/project/sp...
Open API/Swagger File
github.com/openai/openai-open...
OpenAI Documentation
platform.openai.com/docs/intr...
Timestamps:
0:00 - Introduction
0:13 - Demo
0:52 - Creating FlutterFlow Project
2:37 - Setting Up Theme Settings
3:32 - Designing App Page
5:34 - Recording Buttons
8:44 - Waveform
11:14 - Widget Animations
12:55 - Reviewing Cloneable Project
12:09 - Voice-To-Text Custom Actions
16:27 - Chat Completion API Call
19:40 - Stop Recording Actions
20:18 - Adding Multiple Languages
22:34 - Text-To-Voice Custom Action
27:35 - Adding Remaining Actions
28:47 - Timer Widget
31:37 - Final Demo of Working App
Ready to try FlutterFlow for yourself? Start building your app today with a free trial 👉 www.flutterflow.com
Follow us on Twitter 👉 / flutterflow
--------------
FlutterFlow is a low-code builder for native apps, bringing design and development into one tool. With drag-and-drop functionality, you can build pixel-perfect UIs and easily connect your app to live data via Firebase or APIs. Plus, you can add advanced features like push notifications, payments, animations, and more. Whether you build your own custom widgets or write custom code, FlutterFlow makes it easy to bring your app ideas to life.

[@humandesign.commons]

Amazing. I was waiting for this Flutterflow tutorial since Whisper came out 😄 Thank you!!

[@jae.kingsley]

great tutorial, love the pacing of this! please do more

[@KAI-el4ld]

Amazing work thank you !!

[@younessaznag470]

well done ff i was waiting for this video for a long time, now i will go the paid version , good work flutterflow team

[@armankarambakhsh4456]

Can't I run it as test for myself for free?

[@younessaznag470]

@@armankarambakhsh4456 what is the poblem that you had

[@jamiepattersoncta]

This is fire, thank you. People could create generational wealth with this skillset

[@mohanpremathilake915]

Yes! Indeed

[@shabatidjani1354]

nice one when would the image to text own be done ?

[@musumo1908]

Cool project will this work with the GPT4 new super API…the preview one? Assume the chat completions code needs changing?? Thx

[@MeriMAAkiRasoyi]

Great tutorial 👍

[@atsileitumeleng]

Tell me you made that audio up, haha... hilarious.

[@Samuel_Nicole]

Hey @FlutterFlow in the action flow editor there is the: "start and stop audio recording" actions why did you use a custom action instead of these built in actions?

[@humandesign.commons]

Finally!!

[@nguyenquangngoc3993]

Is it safe to add the API key as app state if I want to publish to the web?

[@alltimetech8809]

Can you to do a video about the bluetooth ( ble )connection using a esp32 ? I think that in the tutorial have a mistake , because i did step by step , but it never work😢

[@raparaparapaDum]

min 13:15 everyone gangsta till you NEED CUSTOM CODE

[@scottwallace9498]

I would Like to see a video using Assistants and OCR

[@nathanburley9122]

This is brilliant! I'm a complete noob at all this so I have a bunch of Questions if anyone can help please? 1) I assume I'd be able to deploy this app (or one very much like it) to mobile if I were on the Standard plan via APK? 2) This pubspec dependancy seems to eliminate the need for using Whisper or Google's S2T API - is there a list of other great dependencies 3) Why does the Standard plan only have the same number of API endpoints as the free plan?! I see that you can make API calls with custom actions but I can't quite figure them out, does anyone have any good guidance on this?

[@punishedproduct]

@flutterflow how do we do this for other ai llms? Like huggingface llama etc? It would be cool to see other llms used in the app space with ff integration.

[@tutkinrannan2184]

I get the following error: Refused to load media from 'data:audio/mp3;base64,' because it violates the following Content Security Policy directive: "media-src *". Note that '*' matches only URLs with network schemes ('http', 'https', 'ws', 'wss'), or URLs whose scheme matches `self`'s scheme. The scheme 'data:' must be added explicitly. Is there any fix for that

[@abdulbaset2886]

wow

[@DeLeizard]

Hi, could you help to explain why SpeechToText throws an error "msg: not-allowed" on FlutterFlow Test mode but can run normally on Run mode?

[@vavo1997]

Informative video, but does the app only work on the web platform?

[@FlutterFlow]

In this tutorial, yes. The custom action used to play the audio back utilizes web-specific features. This could be easily adjusted, however, by using a package on pub.dev!

[@armandoortiz3613]

@@FlutterFlow Excellent Tutorial as always! Appreciate all the great content. Can you elaborate on your response about how we can easily adjust the audio playback so the custom function will work on iOS and Android deployments? I see the "//Play the audio audioElement.play();" in the function lines but am not sure what to replace them with and which changes to make. Like what can we use from pub.dev and how would we best insert that into the custom function. I am new to Dart and Flutterflow but these tutorials are a lifesaver. Thanks!

[@AdrianMcMillian]

Following this thread, as I also would like to get this working via iOS and Android.

[@m3gatoile855]

Me too

[@Nolaready]

@@armandoortiz3613hey Armando, did you ever get a fix to this problem?

[@n1Metrics]

Just a heads up that this no longer runs due to issues with speech_to_text

[@Julian-pg9dx]

Thanks, I was wondering why it wouldn't work.

[@MalikaYunus]

How do you fix it?

[@KAI-el4ld]

Just one quick question, how can I add a function to stop the text to speech ?

[@melvinjongen1697]

How can we connect this to a google agenda so it can make appointments and send a notification to a client

[@GrecoFPV]

N8n or make automations

[@yuryarta4832]

I can not deploy my app with your action functions to app store, I'm getting this message: "the app status of version has changed to invalid binary". What should I do?

[@Worganizer]

Can some provide link or explain, how to make it for app version, not just web version

[@marip0sa25]

I get error 401 failure when I tested it can you help me pls

[@MarkSpeaker]

Is there a reason you used a custom package to transcribe the audio instead of the OpenAI transcription service? Could you update this to use that instead? thx

[@FlutterFlow]

The custom package is free with minimal latency (doesn't require an API call), hence the use in this video. The whisper API could be used simply by adding an API call to the project and passing in a saved recording of the user's audio message.

[@PrimeURL.VIP_]

Where these retuned audios get stored? In app, if so how storage manages? What if I want to store it in database?

[@FlutterFlow]

The API returns these are direct MP3s, so you'd need to write an additional custom function or adjust the one used in the video to upload the bytes to Firebase (or another storage bucket) in order to use them later!

[@PrimeURL.VIP_]

@@FlutterFlow got it

[@AdhamFaisal875]

please hot to note and play and download this audio? how to add options maleVoice and famleVoice

[@shabatidjani1354]

hey @ff i have been trying to clone the app but no way what is the problem please ?

[@taha-fd6cr]

must we have chatgpt 4 ApiKey ?

[@Julian-pg9dx]

Can you do an updated version since this one doesn't work anymore?

[@melvinjongen1697]

@FlutterFlow it doesnt response to my question it just says something random how can i fix this ?

[@billybetts5514]

I get the same behavior. it's as if the transcribed audio is not being passed to the api via the [prompt] parameter. when running in test mode in browser I see the below errors: Any help is appreciated. ✌😎 dart_sdk.js:50705 registerExtension() from dart:developer is only supported in build/run/test environments where the developer event method hooks have been set by package:dwds v11.1.0 or higher. dart_sdk.js:29145 Starting text recording dart_sdk.js:29145 Error!: SpeechRecognitionError msg: not-allowed, permanent: false dart_sdk.js:29145 Stopping text recording...

[@jejidyanjackson2868]

so many custom actions, i gonna generate that with AI, Or i gonna first call Buildship and run all the functions i want

[@CodeWonders_]

Do this using gemini api

[@MatthewFusco-wx9pq]

Does anyone have a fix for the issue of it not working anymore?

[@monstarfirst]

I use android stoudio in my flutter project can you tell me the name of this program?

[@ilya_old]

Has anyone successfully compiled this feature to ISO and Android? Because I'm struggling with dart.html.

[@internetisbeautifull]

Same here , please reply if u have somehow solve the problem.

[@AdrianMcMillian]

Following this thread, as I also would like to get this working via iOS and Android.

[@ilya_old]

​@@internetisbeautifull I've asked the developer to rewrite the code without dart.html. When we test it on a MacBook (virtual device), everything is okay. But when I compile it for the App Store, I can't hear the answers. We haven't solved this issue yet.

[@ilya_old]

@@AdrianMcMillian I've asked the developer to rewrite the code without dart.html. When we test it on a MacBook (virtual device), everything is okay. But when I compile it for the App Store, I can't hear the answers. We haven't solved this issue yet.

[@Nolaready]

So ChatGPT replaced the dart:html and uses “Just_audio” and a pub dependency version 0.9.36. But I can’t for the life of me figure out how to link it together.

[@angwaykiam2421]

is there any way for it to work on the app itself without publishing it ?

[@danvanrider5923]

I found a major different between using my iphone and my laptop. Majority of the time it doesn't work from the browser on my iphone but it does on my laptop. Has anyone else experienced this issue? Also, has anyone taken this project and used an AI assistant with passing an assistant ID.

[@AdrianMcMillian]

I also noticed issues when attempting to run on iPhone (via browser). The response message playback seems to be delayed.

[@jamesballantyne9214]

if someone can get a working template of this they will make millions. I have so many use cases but haven't been able to get mic working on mobile apps.

[@nfxberlin3695]

i faced that error while testing the API . how to solve it ASAP please { "error": { "message": "The model `gpt-4-1106-preview` does not exist or you do not have access to it.", "type": "invalid_request_error", "param": null, "code": "model_not_found" } }

[@Juan-Ballesteros]

Won't work for iOS (neither WebApp in iPhone) :( . The Voice-to-text fails so bad. What can I do?

[@nellyanderson8623]

can anyone please educate me why three is no FLutterFlow tutoria on the web to show how to connect a flutterflow app to a web admin backend so that an admin can atleast perform basic CRUD to populate the app we have built in flutter flow ?.. In a real life application, Most app will need a backend to see users orders, upload products etc

[@FlutterFlow]

Try duplicating your FlutterFlow project to create a web-only application. This will allow you to connect the project to the same backend (Firebase, Supabase, etc) but deploy the app on a unique URL that is different than the main app you've published. This would allow you to have two interfaces that connect to the same backend database but could be hosted at app.[your-link].com and admin.[your-link].com.

[@nellyanderson8623]

@@FlutterFlow Thank you for this response. This means i can equally make a web admin panel with futterflow and publish it for web to talk to the same Firebase db as my app to populate/perform CRUD on my app

[@lawrence1679]

@@FlutterFlowI have asked this question several times also, Im not sure why every one who makes a Flutter flow tutorial only focus on the App client side and do not show how to connect it to a backend..You need to show this in one of your videos pls.,. If we can not have an admin panel to populate the app, then do we go into firebase everytime to change things manually ?

[@FlutterFlow]

@@lawrence1679 our team covers these types of videos on the livestream. There are a couple of solid videos on this to get you started: kzbin.info/www/bejne/sISrfGZ9rqijrNk kzbin.info/www/bejne/m5SblKJ8rtCljaM

[@JimmyCampbell6]

does this work if you were to export to mobile given fetchSpeechAndPlay is using dart.html?

[@szymon2030]

@FlutterFlow we have problem with "fetchSpeechAndPlay".

[@tomcuong9588]

I tried to test the API call and it gave me a 404 (failure). The message said that I don't have access to GPT 4. Is this mean that I have to subscribe to Chat GPT Plus? Are there any other way?

[@taha-fd6cr]

Same problem, I am only using chatgpt 3.5 turbo s apikey

[@cryptoairdroppp]

change the model@@taha-fd6cr

[@hamadayahia2152]

Yes, I got that error. The workaround is: API Call -> body -> replace line 8 with: "model": "gpt-3.5-turbo-0301",

[@younessaznag470]

that is the right answer bro chat gpt 3.5

[@akshatgupta4408]

i am getting an error model not found? please help

[@carlanlyortiz8471]

Do you need to be a premium ChatGPT user for this?

[@FlutterFlow]

You'll need to add billing information to OpenAI in order to obtain an API key. This project does NOT require the premium version of ChatGPT.

[@RajeevSharma-ii7fx]

Watched the first 6 minutes. Is this a Text-To-Speech tutorial or a Speech-To-Text tutorial? Asking as i saw you put an input microphone in the container....

[@firmankrisnadi7248]

Can I build this application with free account because I'm newbie

[@ricardoavellan6962]

algun tutorial en español :/

[@jamiepattersoncta]

I'm getting this error when publishing: "Target of URI doesn't exist: '/backend/backend.dart'. Try creating the file referenced by the URI, or try using a URI for a file that does exist." for each of the custom code sections

[@adilet3278]

you copied everything from code, including imports, but in fact you shoudl copy only where is written with comments. Or just delete this import of backend