Build a 3D AI Teacher w/ Next.js, ChatGPT & Azure

Let's learn how to make an AI Language Teacher with Next.js, React Three Fiber, Microsoft Azure Speech SDK, Chat GPT, and custom 3D models made by Fiverr Creators.
Fiverr
go.fiverr.com/visit/?bta=9284...
Use my promo code WAWASENSEI to get 10% off at the checkout process
3D Character Designers:
Beeapoo go.fiverr.com/visit/?bta=9284...
Sahasub go.fiverr.com/visit/?bta=9284...
3D Classroom Designers:
Remigiusz
go.fiverr.com/visit/?bta=9284...
Adityia
go.fiverr.com/visit/?bta=9284...
Deploy the project on Elestio
elest.io/
🚀 Learn React Three Fiber with my complete course ✨
lessons.wawasensei.dev/course...
Final code 🔗
github.com/wass08/r3f-ai-lang...
#threejs #chatbot #reactthreefiber
▬▬▬▬▬▬ Useful L I N K S 🔗 ▬▬▬▬▬▬
OpenAI API Key
platform.openai.com/signup
NextJS Getting Started
nextjs.org/docs/getting-start...
Azure API Key
azure.microsoft.com/en-us/free
Azure Speech SDK
learn.microsoft.com/en-us/azu...
Virtual Girlfriend Tutorial
• How to Build a 3D Chat...
Mixamo Animations
www.mixamo.com/
▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬
00:00 - Intro
00:43 - Custom 3D models with Fiverr
02:30 - Next.js Project
03:28 - React Three Fiber w/ Next.js
08:44 - OpenAI
14:22 - Azure Speech SDK
19:27 - Zustand State Management
25:15 - 3D Html UI
32:47 - Visemes & MorphTargets
35:42 - Teacher animations
37:54 - Final polish
42:25 - Deploy to production
44:34 - Final result
45:40 - ありがとう
▬▬▬▬▬▬ Support the channel 🙏 ▬▬▬▬▬▬
Become a member to get access to awesome perks:
/ @wawasensei
▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
💻 The Discord Community
/ discord
📸 Instagram :
/ wawa.sensei
🎎 Facebook :
/ wawasenseiyt
🐦Twitter :
/ wawasensei
🐦TikTok :
/ wawasensei08
💻 My website :
www.wawasensei.dev

[@itsanishjain]

You are underrated. I never saw anything like this.

[@WawaSensei]

🙏 Thank you so much, glad you like it!

[@trifonstatkov477]

Kudos to the man that makes the coolest stuff with React Three Fiber in KZbin. Simply incredible!

[@WawaSensei]

😍 such a pleasure to read this, thank you 🙏

[@gaijinshacho]

You make 3d web dev look so easy! Great job! 🎉

[@stain5570]

You deserve 100 times more views bro, this is some next level stuff, I haven't seen anything like this on youtube

[@animatronext]

Wow! These are amazingly done tutorials. I got hooked until the end. Kudos!

[@WawaSensei]

Awesome, thank you! 😊

[@kushranadaioi685]

Well done bro, I already have your course (yet to finish it). But this is an artwork.

[@the_rings_of_saturn]

A very underrated channel! You are doing the coolest stuff with R3F. There are almost no R3F projects on KZbin. Keep up the great work! 😊

[@WawaSensei]

More to come! Thank you so much! 🙌

[@guacamole456]

Great content and great ideas! This channel deserves more subscribers; please spread the word.

[@WawaSensei]

Much appreciated! 🥰

[@user-is8vt8rc2g]

I enjoyed this amazing course. thank you

[@WawaSensei]

My pleasure!

[@mdalishanali3909]

Bro, your content is really high and unique quality ❤. It’s always help to learn something new. Thanks man!

[@WawaSensei]

A pleasure to read, thank you! 🙏❤️

[@628sonu]

this is dope,. kudos for building this

[@WawaSensei]

🥰

[@genshian]

No words... Just feeling grateful sepai exists.

[@WawaSensei]

Owww, thanks buddy 🙏

[@mattkkma]

Awesome stuff!! Azure Speech SDK is a a great resource. Azure TTS returns visemes and also the other standard facial blendshapes. Did you also apply these other blendshapes to the face? I found they interfered with each other an distorted the mouth a lot. I'm hoping you might have nailed than and can share your approach.

[@stamatiskourtis9653]

Really great stuff once again!

[@WawaSensei]

Thanks again! 🫰

[@hafiedzmada]

Awesome work, keep it up!

[@WawaSensei]

Thanks, will do! 🙏

[@unhealthytomato3219]

Your work is really inspiring!

[@WawaSensei]

Happy to hear that! Thank you very much 🙏

[@b166er4]

wonderfull work ... je suis impressionné par le travail que tu fais et les projects que tu sorts .. depuis 3 ans thx/merci

[@WawaSensei]

Merci beaucoup, trop bien que tu sois là depuis tout ce temps! 😍

[@Dan-Levi]

This is pure awesomeness! Got me started on my jarvis clone. 🙂🙂👍

[@WawaSensei]

Let's go, happy to help!

[@inteligenciafutura]

The idea is very excellent to put into practice.

[@WawaSensei]

Thank you! It's an idea I have for months, it was so cool to be able to have 3D models done exactly for this purpose that I can use & share 🔥

[@Assoehosselaar]

Amazing tutorial, and very unique. Thanks for this.

[@WawaSensei]

Many thanks to you! 🙌

[@malickgm507]

Covered so many things in one video. Even though some things I couldn't understand lol but still man you are awesome! Great Stuff

[@WawaSensei]

Awesome, thank you! Don't worry if you don't understand 100%, you can come back at it later as your knowledge grows and understand more and more 🙏

[@mr.daniish]

Wow! i am blown away.

[@WawaSensei]

🥰

[@theSlavenIvanov]

Great work!

[@WawaSensei]

🙌

[@boshen3953]

🔥🔥Amazing Video 🔥🔥

[@WawaSensei]

Thanks a lot Rick 😍

[@user-qb7gk4ct4b]

Thanks you for your efforts bro, Really helpful 😇

[@WawaSensei]

So nice of you 😍

[@kihwanlee5617]

Wow! Amazing!

[@WawaSensei]

🤗

[@nicolasportu]

Outstanding!

[@WawaSensei]

Thank you kindly!

[@jameszokah]

Woow what an amazing video

[@WawaSensei]

Thank you so much 😀

[@julianvelez6563]

Wow bro awesome work bro you are the Best bro ❤❤

[@WawaSensei]

Thank you so much 😀 N°1 Support! 🔥

[@ggwp185]

this is amazing.

[@WawaSensei]

🙏🙏🙏

[@fitzladams]

I am considering buying his course.❤❤❤

[@WawaSensei]

🙌 If you do, wish you'll like it!

[@ryanchen1793]

good tutorial, building a web based virtual human is much easier than before by leveraging AI capability and three.js render engine.

[@WawaSensei]

Thank you! Yes so true, very simple, we’re so lucky to live in that era 🤖

[@draftermyself]

WOW! 😮

[@salimnegan7289]

You should use Mistral, you gonna reduce your cost and maximise your gain. Better solution is to embarq a phi model :) But thank's for Three FIber tutorial, as a react developer with an AI Background, I am learning all that thing possible with three fiber

[@nsd244]

Great!

[@WawaSensei]

You are great!

[@nickysalazar1418]

Voce é muito foda gentee, muito bom video

[@WawaSensei]

Obrigado! 😍

[@lettechtutorials]

Very Good Great

[@WawaSensei]

Thank you so much 😀

[@lettechtutorials]

@@WawaSensei Welcome I am available for paid and free assistance for your channel

[@lincroyablemrbebe8390]

realy goog jobs

[@WawaSensei]

🙏 Ow, thank you!

[@mninounassim]

Amazing ! React native please

[@WawaSensei]

You might be surprised in a few hours 🤭

[@smartjackasswisdom1467]

Can you make a tutorial on how to use Vroid characters (.vrm format) on Three.js? I'm a beginner with three.js and I've been breaking my head trying to figure it out.

[@WawaSensei]

I often see this request! Actually, I tried a bit a while ago without success. I think it's more a 3D modeler topic than a Three.js issue. You'd need to convert your VRM ideally to .glb

[@yalamanchilivenusai8874]

Hi I am from India, your videos are amazing , it would be great if you can add pricing based on countries. The problem here is I am unable to use Open AI api keys on my free account, free trial no longer works here but this video is great, thanks!

[@WawaSensei]

Thanks a lot! I think even if you put your card with OpenAI, before it costs you something you need to use it a lot. With all the tutorials and project I made with mine, I was never charged (no intense use from users, but still)

[@piggybank1385]

🔥🔥🔥

[@WawaSensei]

🙌🙌🙌

[@ShivamKumar-cu3lb]

What was the source for animation files ( fbx ) apart from miaxmo is there any other source which i could use for customised animation?

[@WawaSensei]

All fbx in this video are from Mixamo, you can try learning to create your own animations with Blender (for ex: kzbin.info/www/bejne/iputfXyre7B0aJI) Or search for motion capture software

[@pradyumncs]

Can you do just one in react native expo . Just one . No tutorial at your scale in react native

[@WawaSensei]

Hey, might do a lesson about it yes!

[@alexxarasho1138]

Can you make a video explaining how to use .vrm models from Vroid studio inside three.js? Is it even possible? I've tried so many tutorials and documentation trying to achieve this but always fail miserably. I even ended up picking Unreal Engine 5.3 to do this just because I couldn't find a way to do it with three.js and Typescript.

[@gaspacho961]

I'm working on just that. I'd recommend converting your .vrm to .glb (just change the extension) for starters. Importing other model doesn't seem so easy, but I'm trying and I hope you succeed as well lol

[@WawaSensei]

I tried briefly without success a while ago, might try again in the future 😔 If anyone has a good solution I'm all ears

[@codersandy]

Hi, Can I know what are the requirements you sent to the Fiver artist for the avatar model creation. I am trying to implement models collected from other sources (not ready player me) and those are not working. I will be very thankful for your help. Thank you for the great tutorial.

[@WawaSensei]

Hey, here were my requirements: - Rig has to be compatible with Mixamo (no need to attach animation, I will do it programmatically in my project) - The following facial blendshapes: eyes closed, eyesLookUp, mouthSmile, and the 21 visemes attached as a picture (to make the character speak) The attached picture was a screenshot of this website -> learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-speech-synthesis-viseme?tabs=visemeid&pivots=programming-language-javascript

[@codersandy]

Thank you so much ❤❤❤

[@fedev1312]

thanks , reactjs three are meta trend 2024 cuz build VR web/app UX/UI

[@WawaSensei]

Thank you! I definitely can, do you have any XR app ideas?

[@fedev1312]

@@WawaSensei i have and i'm learning reactjs three on your tutorials. Then I will do some like products, virtual fairs, online virtual games.

[@salaheddineelmahjour8956]

💯💯

[@WawaSensei]

🔥🔥

[@SpeakUpIst]

Hey Wawa Sensei ! Thank you vey much for the video and app. I followed along it works perfect on windows also in mobile view but when i opened in iPhone 14 Pro Max The Board is half hidden and buttons can't be clicked. I tried to fix it then tried your originial finished github code. Same problem exist there. Also added to issues part on github and attached image. Tried a lot of time to fix but couldn't. Can you help?

[@WawaSensei]

Thanks a lot, I'll have a look at the iPhone issues and push the fixes 🙏

[@SpeakUpIst]

@@WawaSensei Thank you very much. This tutorials opened a door for me.

[@WawaSensei]

@@SpeakUpIstyou're welcome! I've updated the source code, on iOS with the distanceFactor of 0.5 on the HTML, it caused issues. Setting it to 1 and doing the necessary adjustments fixed it 🙌

[@Ibrahim-oc5ql]

wOOOOOW SESNSEIIIIII LFG

[@WawaSensei]

Thanks Ibrahim!

[@anheminhuc6325]

Hi I'm newbie, Could you share the vscode ext that can view the model file?

[@WawaSensei]

Here it is marketplace.visualstudio.com/items?itemName=degreat.3e 🙌

[@anheminhuc6325]

@@WawaSensei Thank you so much 😘

[@AniLYadav-ln2ty]

Hey.. thank you.. I watch your all videos.. it's amazing ❤ and i want to connect with you so how can I connect with you ? I we development and I want to learn more about 3D . I'm From Nepal here is no any platform like react in 3D so I want to connect with you and I want to do your your that's gives my learning growth.. so plz help me.. I'm from small country and here is no that much good opportunity .

[@WawaSensei]

Thank you very much! 😍 Best place to connect is publicly in the Discord lessons.wawasensei.dev/discord Unfortunately from now I don't do pricing based on locations, I might do a discount on the course for the launch of the new shader chapter (expected before end of the month)

[@subhamkundu8234]

Sensei please release a full react three fiber course for free I don't find a single full three js course in you tube .

[@WawaSensei]

Hey, some part of my course are free and all lessons include a free preview: lessons.wawasensei.dev/courses/react-three-fiber

[@gaurav20ist]

Please make project ai assistant avtar with speech recognition and reply it .

[@gaurav20ist]

Please help to make project, if we ask the self clone avtar, what is your name then it will play the mp3 - my name is..... , speech to speech response with mp3 or wave data pre saved with expressions and lip sync

[@WawaSensei]

Maybe in the future, it's in my ideas list

[@skylake315]

i cant use api that was mean i must buy the api $5 price ??? i didnt have dollars

[@WawaSensei]

Unless they based it on your location, there's a free trial, you need to register with your credit card but it won't be used.

[@bishal.dangol]

Can you share code and the model through git?

[@WawaSensei]

Everything is in the description

[@bishal.dangol]

@@WawaSensei ok

[@alexis.brissard]

C'est bô

[@WawaSensei]

Hehe marchi Brissou 😍

[@likwaichan6208]

Can u be my sifu

[@WawaSensei]

My pleasure! 🧑‍🏫

[@veerasinghindia]

Is that your real voice ? It seems you are putting too much effort to sound like this ?

[@WawaSensei]

I’m a bot, it’s AI 🤖

[@veerasinghindia]

@@WawaSensei ha ha felt so ! Thanks for the content though

[@burhaann]

Amazing Video as Always but i want to remind you about one of your other video, it has a bug kzbin.info/www/bejne/pIKsaYiEmpuWnKMsi=zTSyUnbK7JqqKIub [BUG REPORT] It has a bug, if there is more then 1 players, all players other then the host get stuck, i tried commenting on that video, but for some reason my comment used to get removed You can recreate this bug, by opening your deployed url and try to join from another browser tab, the second player will get stuck in one place!

[@WawaSensei]

Hey thank you! Weird about the deletion of other message (while it seems it worked here) can you share the issue on the Discord? lessons.wawasensei.dev/discord