Azure OpenAI Deployment Types and Resiliency

[@NTFAQGuy]

Hey everyone, let's explore Azure OpenAI and all the options and resiliency considerations! Please make sure to read the description for the chapters and key information about this video and others. ⚠ P L E A S E N O T E ⚠ 🔎 If you are looking for content on a particular topic search the channel. If I have something it will be there! 🕰 I don't discuss future content nor take requests for future content so please don't ask 😇 🤔 Due to the channel growth and number of people wanting help I no longer can answer or even read questions and they will just stay in the moderation queue never to be seen so please post questions to other sites like Reddit, Microsoft Community Hub etc. 👂 Translate the captions to your native language via the auto-translate feature in settings! kzbin.info/www/bejne/rGbFZmZjhcx4o6s for a demo of using this feature. Thanks for watching! 🤙

[@Dikimkd]

I am LOVING these AI content videos!

[@NTFAQGuy]

Glad you're finding them helpful!

[@RistoRatilainen]

This was very informative “summary” about service in question design options.🤝

[@MrAmgadHasan]

Thanks so much for covering this. I especially loved the fact that the azure openai resource is regional even if the deployment is global. Just one thing about prompt caching: we cache the output of the tokenization step and the prompt processing step as well (calculating the k-v cache).

[@NTFAQGuy]

Good info on the prompt caching. Thanks!

[@DavidManouchehri]

Are you seeing prompt caching actually working in production? I have yet to see a single request for gpt-4o or gpt-4o-mini work with caching on Azure OpenAI.

[@elprofesornet8897]

Great video, thanks John. I can now translate this information to my customers.

[@VirtualPackets]

Really nice breakdown and whiteboard session, makes perfect sense, thx again John.

[@NTFAQGuy]

Glad you liked it!

[@LifeisbetterwithaMalinois]

Thank you sir John😊

[@NTFAQGuy]

You're welcome 😊

[@akhil.varierm1899]

First comment.😀As always, amazing content and presentation. Thanks John.

[@NTFAQGuy]

Appreciate you checking it out!

[@denver-reed]

I'm a bit confused on the intelligent routing plus the need for APIM. If a global deployment has intelligent routing to figure out which endpoint is best given utilization + latency, how does APIM fit in?

[@NTFAQGuy]

Resilience for endpoints, ptu to paygo etc as I talk about in video