Asterisk Google Cloud Speech API Integration

[@mada7ebnana]

Excellent foundation and presentation .. very brilliant .. I will try to spare sometime and try to replace Google's service with one of the opensource LLMs which will definitely cut down the overall costs .. keep it up buddy .. brilliant effort and surely appreciated! 💪💪👍

[@omidmohajerani]

Thank you. That would be great idea. What opensource LLM do you prefer? For connection you can take a look at vosk app as well that has an asterisk app.

[@hidayatnst]

Thank you for your hard work! I learned a lot from this channel.

[@omidmohajerani]

Hidayat Hidayat Hidayat . Thank you . Come and join the team in Germany :)

[@haotran9025]

Thank you for your assistance. It is working, but how can we make the system stop playing audio when the caller starts speaking? If the audio continues to play while the user is talking, it doesn't provide a good user experience.

[@omidmohajerani]

yes you need to work with dialplan. search about SpeechBackground and also there are multiple ways to do it Silence detector is part of the DSP: github.com/asterisk/asterisk/blob/master/include/asterisk/dsp.h Example implementation: github.com/asterisk/asterisk/blob/master/funcs/func_talkdetect.c The res_speech module: github.com/asterisk/asterisk/blob/master/res/res_speech.c The res_speech_aeap module: github.com/asterisk/asterisk/blob/master/res/res_speech_aeap.c it’s up to you to piece it all together

[@simonepalomba4455]

Hallo Omid. Thanks for your video....never been able to change language...en-US works smoothly....but everything else not.....any advice ?Thanks

[@omidmohajerani]

Hey, I have not used it much and for other languages . Nowadays i usually use signalwire or twilio . So i send the call via sip to them and then i use their apis …

[@simonepalomba4455]

@@omidmohajerani ok…for Italian you need to modify languages.js and provider.js the first const supported = [ “en-US”, “it-IT”, ]; the second static languages ​​= [ “en-US”, “it-IT”, ]; And const DEFAULT_LANGUAGE = “it-IT”;

[@omidmohajerani]

@@simonepalomba4455 Thank you for the info and update

[@aminfateminia8643]

Hey Omid, Is there a way to transcribe the calls on the server itself instead of uploading them to Google Bucket? I could have used the service, but it seems there is a maximum length limitation of 1 minute for local usage.

[@omidmohajerani]

Hi, I believe you mean a local speech to text system. You can run a local ai speech to text like open ai whisper on a local machine. Ofcourse you need proper hardware for that to have acceptable speed.

[@aminfateminia8643]

@@omidmohajerani The problem with OpenAI is that you also need a good GPU on the machine, which is not always feasible. Anyway, I figured out how to overcome this limitation.

[@CodeWithAqeel]

Thanks for explanation. Now how can we achieve the two way communication like two zoiper users are talking to each other then how can we receive speech to text of both

[@omidmohajerani]

For your scenario, you need to stick to the channel and get steam from the channel. There are multiple ways to do it such as using rtpengine or sniffing method.

[@CodeWithAqeel]

@@omidmohajerani will be waiting impatiently. Thanks

[@omidmohajerani]

@@CodeWithAqeel if you are one of my official Patreon users you can request the video there. Based on the vote there I usually set priority for creating videos www.patreon.com/omidmohajerani

[@ZJoJo]

Thanks for sharing Omid , I tried to apply it on my asterisk but I get below error : "Unable to connect websocket client to 172.0.0.1:9099: Connection timed out"

[@omidmohajerani]

Did you run the node js app as i explained ? When you ran it did you get any error?

[@ZJoJo]

@@omidmohajerani Yes, I ran it with no error , as you can see below : "root@IPCC-Asterisk-Test:/usr/src/aeap-speech-to-text# ./index.js Server on port '9099': started listening"

[@omidmohajerani]

@@ZJoJo but this does not show any error. It shows it listening without any problem on port 9099. You need to make a call and then check here if you have any errors and provide that.

[@ZJoJo]

@@omidmohajerani after I make the call here nothing showing at all.

[@ZJoJo]

the error only showing when I navigate to Asterisk -rvvv : Connected to Asterisk 20.6.0 currently running on IPCC-Asterisk-Test (pid = 75635) == Using SIP RTP CoS mark 5 -- Executing [550@internal:1] NoOp("SIP/phone[phone]", "") in new stack -- Executing [550@internal:2] Answer("SIP/phone[phone]", "") in new stack -- Executing [550@internal:3] SpeechCreate("SIP/phone[phone]", "my-speech-to-text") in new stack [Mar 7 12:33:54] WARNING[81625][C-0000000d]: tcptls.c:656 ast_tcptls_client_start_timeout: Unable to connect websocket client to 172.0.0.1:9099: Connection timed out

[@mehta.arvind-g4s]

I need to run KOHMP. How to run that ? Please explain.

[@omidmohajerani]

I have run it before and know how to but they have a support . Why you dont contact them ?

[@mehta.arvind-g4s]

I pinged Them but No Reply@@omidmohajerani

[@ChristopherGitonga-tp6kq]

Hallo Omid. Thanks for sharing. Your channels have taught me alot. Any chance you have vicidial text to speech intergration too.

[@omidmohajerani]

I have one with asterisk . Its the same as vicidial is using asterisk

[@ChristopherGitonga-tp6kq]

@@omidmohajerani thanks for your response. if you could kindly share the link to this i would be grateful.

[@ChristopherGitonga-tp6kq]

did you manage to get this tutorial uploaded. we only see speech to text

[@omidmohajerani]

@@ChristopherGitonga-tp6kq Hi Sorry. you are right. It's not uploaded yet. I didn't have much time recently to edit and publish.

[@witchdoc-xp]

First Comment just for the algorithm. Btw is the text to speech with Microsoft also so easy? Or which services did you prefer?

[@omidmohajerani]

Thanks Karsten for the comment. Yes basically I find out 2 methods to do it. The First method was to record a the speech into a wav file and send the wav file to Microsoft or google as they accept wav format that in Asterisk usually is done via an AGI script. But i noticed it has significant delay in comparison to second method that we stream voice to them. When asterisk introduced External Media and aeap then it was easier to do send stream as via ARI we can tell asterisk when to route the media.

[@witchdoc-xp]

​@@omidmohajeranifor debugging it would also be nice to know if you can temporarily save the stream (to see if the input is proper or not). The npm stuff I would definitely push into an docker container

[@marcinborowik7225]

Hey Omid , i am getting this: On Asterisk: -- Executing [550@all_peers_3:1] NoOp("PJSIP/phone-0000002c", "") in new stack -- Executing [550@all_peers_3:2] Answer("PJSIP/phone-0000002c", "") in new stack > 0x7f0ff805e740 -- Strict RTP learning after remote address set to: 192.168.0.4:8000 > 0x7f0ff805e740 -- Strict RTP switching to RTP target address 192.168.0.4:8000 as source -- Executing [550@all_peers_3:3] SpeechCreate("PJSIP/phone-0000002c", "my-speech-to-text") in new stack [Aug 10 13:27:49] ERROR[4054]: res_aeap/aeap.c:204 raise_msg_handler: AEAP (0x7f0fd800f8e0): Cannot read property 'streamingRecognize' of undefined -- Executing [550@all_peers_3:4] SpeechStart("PJSIP/phone-0000002c", "") in new stack On server: root@asterisk:/home/voip/aeap-speech-to-text# node index.js Server on port '9099': started listening Server on port '9099': client connected message: {"request":"setup","version":"0.1.0","codecs":[{"name":"ulaw"}],"params":{"language":"pl-PL"},"id":"381b2318-1f8f-4aae-b7e8-12d0b8db702f"}

[@marcinborowik7225]

The problem has been solved; updating Node from v12.22.9 to v18.20.4 helped.

[@omidmohajerani]

Yes that la node version problem . Goodluck

[@av-dm3ny]

Hey Omid , i am getting this ... Executing [550@office-phones:1] NoOp("PJSIP/bob-softphone-0000000d", "") in new stack -- Executing [550@office-phones:2] Answer("PJSIP/bob-softphone-0000000d", "") in new stack > 0x7f2518035d70 -- Strict RTP learning after remote address set to: 10.42.0.210:64763 > 0x7f2518035d70 -- Strict RTP switching to RTP target address 10.42.0.210:64763 as source -- Executing [550@office-phones:3] SpeechCreate("PJSIP/bob-softphone-0000000d", "my-speech-to-text") in new stack -- Executing [550@office-phones:4] SpeechStart("PJSIP/bob-softphone-0000000d", "") in new stack == Spawn extension (office-phones, 550, 4) exited non-zero on 'PJSIP/bob-softphone-0000000d' i think asterisk is not calling my websocket endpoint. can you help here?

[@omidmohajerani]

It may be because it's not running and the asterisk can't connect to it. Did you use the updated config from my pad file in my Patreon? That should work as another user just used it a week ago.

[@av-dm3ny]

@@omidmohajerani the node server is listening properly on port 9099. "Server on port '9099': started listening" .But i am using asterisk v21.Is that a problem here?

[@omidmohajerani]

@@av-dm3nyI dont think so. Asterisk 21 should be fine but you can give a try on another version if you are sure you have done the steps without problem

[@omidmohajerani]

One of my users had problem with node ja version . You can test that one as well