Text-to-Speech ElevenLabs Plugin for TouchDesigner + Whisper, ChatGPT, and MediaPipe Integration

In this tutorial, you'll learn how to use the new ElevenLabs text-to-speech plugin for TouchDesigner. We'll also walk through a project creating a voice-activated 3D talking character by integrating Whisper for speech-to-text, ChatGPT for generating dialogue, ElevenLabs for text-to-speech, and MediaPipe for motion tracking to create an interactive experience.
Link to project file + Plugin on Patreon: / tblankensmith
Previous Tutorial on ChatGPT and Whisper: • GPT-4, ChatGPT, and Wh...
Noto the Talking Ball's Tutorial: • Talking ball Starboy i...
MediaPipe Plugin Tutorial: • Face, Hand, Pose Track...
0:00 Overview
3:11 ElevenLabs Plugin Overview
7:22 Connect ChatGPT to ElevenLabs
10:11 Configure Whisper with ElevenLabs
17:40 Add MediaPipe Face Tracking

[@phule5966]

extremely underrated channel. Thank you!!!

[@RetropunkAI]

omg this so so good LOL! Great use case... I assume the sour cream is under the hat?

[@dvdgoldberg]

yea ! you made it :) great work

[@user-rz4re7io6k]

Hi, I'm trying to change some parameters in your text-to-speech plugin to suit my project, but it won't work after I unlock script2, I wonder what's the point of locking script2?

[@blankensmithing]

Locking the script2 CHOP allows you to programmatically write data into the ScriptCHOP. The audio data gets streamed in as chunks of data and I wanted to have the lowest latency so that you can start playing the audio the second it starts streaming instead of waiting until the whole thing finishes downloading.

[@user-rz4re7io6k]

​@@blankensmithing Oooh, thank you very much for your answer, I kind of understand that you wrote a script that reads elevenlabs and streams back the audio and put it in scriptchop, and when scriptchop locks, the script freezes in TD，Like bg1TOP locked？