Generative AI into ANY .NET App with SemanticKernel

Check out my courses at dometrain.com
Get the source code: mailchi.mp/dometrain/f_hqglt_2e8
Become a Patreon and get special perks: / nickchapsas
Hello, everybody. I'm Nick, and in this video, I will show you how to get started with adding LLMs and Generative AI into any .NET application using C# and the SemanticKernel library.
Workshops: bit.ly/nickworkshops
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: github.com/Elfocrash
Follow me on Twitter: / nickchapsas
Connect on LinkedIn: / nick-chapsas
Keep coding merch: keepcoding.shop
#csharp #dotnet

[@Azagatoth]

Ideally I would want to have a local LLM with our training material/documentation, so the users can access that

[@HenningKilset76]

You can have a remote LLM and use the rag pattern to make the LLM aware of relevant documents based on user queries.

[@Azagatoth]

​​@HenningKilset76 Remote LLM is not an option. We have customers that are completely off the grid due to security.

[@HenningKilset76]

Ok - get that. A combination of Semantic Kernel and Kernel Memory can connect to and utilize for instance a local Llama or Phi model as well.

[@georgepagotelis]

The answer you're looking for is Microsoft Copilot 365 (only seen demo online) and there's an copilot developer kit you can "buy" ie. another 100 subscription and do exactly that. Was thinking this while watching too btw.

[@francoislepron2301]

Is it possible with Mistral and train it with my data ?

[@paulguk]

Would be interested in follow up videos that explore interacting with data (eg via SQL) to obtain data relevant to questions asked by the user.

[@MaQy]

Semantic Kernel makes it easy to use RAGs with Azure Search and other DBs. You can also build chat completion tools that the model can choose to call. The example you are going to find everything is a tool that provides the weather.

[@lucaciandrei]

Interesting, but costly.

[@LoganDunning]

Agreed, calling Open AI Apis ditectly is HUGE cost savings compared to Azure Open AI.

[@cheronecom]

I should note, ollama doesn't require a GPU, but you will want to use a small model if you use CPU - like tinyllama or phi3:mini

[@geraldmaale]

@@LoganDunning Check again, both are the same. (input = $0.005/1k tokens, output = $0.15/1k tokens)

[@rogeriobarretto]

It all depends on the underlying AI Service you want to use, SK works as an abstraction regardless of the service you are using. So if you decide to use it for Local Models you have virtually has no cost.

[@HenningKilset76]

@@LoganDunning It's the same price. GPT-4o 1.5 cents / 1000 input tokens and 3 cents / 1000 output tokens.

[@alfany6252]

It would be better if you told about running local LLM with C#

[@nickchapsas]

It’s coming

[@KibbleWhite]

@@nickchapsas Yes please buddy, loving this insight so far! 💖

[@irql2]

Its very easy, just get LM Studio. It exposes an API endpoint you can call in C#. This is what Nick will probably show in the next video because its the easiest to do by far imo.

[@alfany6252]

@@irql2 I want it to be only in my app. I dont want to run other program, run server...

[@maskettaman1488]

It wouldn't be "better", running locally is a totally different scenario for a totally different use case.

[@HenryETaylor]

Directly exposing AI features to my customers would only make sense if it generated higher revenues from existing clients or accelerated the sales cycle (allowing me to grow a bigger customer base), but integrating AI features into the tools I write for myself to generate synthetic data, categorize and analyze server traffic and cloud costs... Yes! That I will start doing immediately!

[@nitrovent]

The most fascinating feature of the semantic kernel for me was the kernel plugins that allow you to give the AI access to local function to reach the goal you specify in chat. You decorate your plugin functions with a description in plain text and the semantic kernel uses it to orchestrate function calls. This way you can access local data that is highly dynamic (local time, user's todo list, whatever) but it will also call functions to create content or control the app. That really blew my mind.

[@danclarkeuk]

Nice. I've got some dotnet code that's interacting with OpenAI via their REST API - but I might convert that over to Semantic Kernel after seeing this.

[@danclarkeuk]

Well that was easy! I've just converted it over and it took about 5 minutes and just worked! Thanks for the video! 😊🙏

[@vincenthamelin6115]

@@danclarkeuk Although your post is relatively new, have you seen a cost difference? I've read that using Azure OpenAI Service is MORE expensive than using OpenAI APIs directly. What are your thoughts?

[@danclarkeuk]

@@vincenthamelin6115 Ah sorry - I probably wasn't clear about what I'd changed. I haven't switched to the Azure OpenAI service. I just changed my implementation from talking to OpenAI manually via their REST API, to instead using the Semantic Kernel nuget package.

[@RoySalisbury]

Would be cool to see a demo on feeding it some SQL table creation statements or some c# objects (perhaps your MovieDB example) and have it generate the code for you. Then ask it to do thinks with the code like "Add the necessary error handling".

[@sergeyz.5845]

didn’t know about that lib. Thank you

[@zachemny]

Also an example of embedding documents with SK into a local model would be very interesting.

[@jorgeserrano4226]

@Nick can we use same code when targeting OpenAI instead of AzureOpenAI ? if not, what would be the diffrence ?

[@TheMannihilator]

Thank you for the demo. Maybe you could use another LLM instead of OpenAI stuff.

[@Chatlan]

Thanks for the video. It would be interesting to see how to use RAG as well

[@nick066hu]

Is there any reason why should we use OpenAI thru Azure ? The Open AI api is also directly available. Can someone with some experience with both pls help me compare these two approaches: Pro/Con /Pricing

[@DaminGamerMC]

I have some experience with Azure Open AI. By itself it is also very very cheap. However, as soon as you want to add you own data (like for making a support service for a company) it gets very expensive because you need AI search. A big issue is that you need to be allowed to use the service by Microsoft meaning you might have to wait between a week and a month to really be able to use it.

[@MetehanG]

For my scenario, my company already has access to Azure and we have some free monthly Azure credits. These credits also available on Azure Open AI so it was basically free for us. Why would we pay for OpenAI API while we have that? Other than that, there is practically no difference as far as I know.

[@ThaGuus]

Pro of Azure OpenAI: Runs on Azure so more "secure", Already known SLA, VNET support & private endpoints, if you want to "train it" then the files are on Azure which is a pre. Con of Azure OpenAI: You need to have a business Azure Account with domain and request access, Streaming doesn't work fully like it will stream in parts but not letter for letter like the normal OpenAI API, It's slower

[@PticostaricaGS]

Azure OpenAI is designed having data security in mind. And has easy integration with other Azure AI services. So, for companies internal tools, Azure OpenAI is more suitable.

[@nick066hu]

Thank you to all who replied and also those who are going to. For a proof of concept evaluation I might try the direct OpenAI api first. (I am not very experienced in programming Azure, learning two things at a time might be unnecessarily steep, although the free monthly credits are also tempting) My end goal is to program a technical support service assistant, with knowledge from uploaded manuals and other documents. First in a chat-window text based, in some later phase in voice mode. @DaminGamerMC What do you mean by 'allowed ....and wait a week/month' ? Am I right with assuming, that in this scenario I pay for the uploaded (they call it 'embedded', if I got it right) documents, - I need these the AI assistant find the replies in -, only once, per tokens in the document, and for all subsequent (customer) queries I pay only for the tokens actually in the query and the replies?

[@sushilb7994]

Very nice!

[@mingyingxue6815]

hi, Nick, what is the ide you used in this video? tkx

[@abdulkadirbener]

Gpt4 supports some training functions with limitations. Is there a way to upload our knowledge and train the AI with SemanticKernel?

[@HenningKilset76]

Yes - I'd look into the combination of Semantic Kernel and Kernel Memory - another MS open-source library.

[@tyomidi]

Would have been cool it is was with Llama but I dont see any good way with Dotnet?

[@Craxmerax]

How would I use this to stream the response to a frontend application and get the same sort of feel where the chat bot is writing in the response like in the CLI?

[@ferd1775]

Yield return

[@ferd1775]

You could also use llama3 locally and ask it 😉

[@neociber24]

AsyncIterator?

[@ferd1775]

@@neociber24 oh yea. Forgot all about that, I have no use for it generally. Way better solution

[@bogdanbanciu1781]

I also posted a tweet about the behavior of await foreach while building an API to do some AI work. Did you try using await Task.Yield() instead of the Delay(100)? Something fishy is happening over there

[@alexanderpoplooukhin7448]

What's a benefit to use chatgpt through MS Azure instead of using chatgpt's API?

[@francescogranozio2780]

could you make a video that shows the “chat with your files” feature?

[@igorsolomatov4743]

Why not use existing ai agent frameworks here? I think the most critical is how do we get grounding of ai. For example I want it to be able to run my functions. So that it will actually integrate

[@Grumpicles]

That's great. But how much does it cost?

[@shabanelmogy7912]

we need big tutorial about blazor and what your opinion in it ?

[@Kotz_en]

Does this work with Ollama, or is it just Azure?

[@Luke-me9qe]

Yes. You would need another connector. And it is available

[@Sztormus]

Cool video as always! But now we want a stream from World of Warcraft 😂

[@nickchapsas]

This has happened already when Dragonflight launched 😂

[@JustArion]

@@nickchapsas Nick the Gamer. Would love to see more of those

[@davidjsutherland]

We live in a wild and crazy time.

[@FraserMcLean81]

I've been playing around with Azure Open AI but was just using the Azure.AI.OpenAI packages. Didn't know about this Symantic kernel thing.

[@neociber24]

This make me notice I have never streamed content to a client side app in C#.

[@craigmcinnes1212]

Maybe I have this VERY wrong, but what has this to do with Semantic Kernel? I thought SK was about providing automation to an AI, so that it could invoke functionality within your app. example, my app is a ride booking service, so I provide a "request ride" "feature" and now the general AI knows about that specific "ability" and how to invoke it within my app during a general chat style session?

[@etiennebaudoux135]

You can do exactly this with Semantic Kernel, but Nick only scratches the surface and likely focused on the basic ChatGPT conversation because that's probably what people are the most familiar with when it comes to AI nowadays.

[@irql2]

Honestly all of that can be done without the Semantic kernel. You can define functions in your app and tell the LLM they're available to call. They call it "Function calling" because they're really good at creatively naming stuff lol.

[@KarmCraft]

How to add your domain logic/knowledge?

[@nickchapsas]

I’ll show that in a future video but it gets pricy

[@irql2]

@@nickchapsas Will be interesting to see what you mean by pricy. The technology to do it is openly available.

[@macoson]

Recommend Autogen which is also available for .NET

[@andreyz3133]

WoW streams wen ser?

[@T___Brown]

I worked with SK like 4 months ago. I couldnt use because of so many bugs. Maybe better now but I have already gone to python.

[@Biker322]

We have a cloud platform , clients all scream we want AI. But none can tell me exactly what they want AI to do. Got. POC working with aws bedrock, but still, can’t think of a single use case for it. Our clients want to say they have something AI, seems to pretty much be it.

[@Radictor44]

Where's the kill switch 😂

[@vyteniskajackas7579]

Oh I wish you would have said for AI generated title at the start "this title is terrible" but still use it on this video as a joke

[@mihaimyh]

WoW FTW!

[@scf0carbonete]

It's great. Sugestion made a Maui, Onxx, PHI-3 small or Gemini nano , multiple OS's example.

[@devincarlson8866]

Someone has been playing Cataclysm Classic recently and misses Wrath haven’t they…

[@nickchapsas]

🥲

[@ronnyek4242]

Not that open implies free... But I find the trend of hosting AI models and charging fees for consumption problematic. Most of the examples I've seen really couldn't be incorporated into real world apps without losing money. I hate that Microsoft kind of pushes people to cloud vendor lockin. I see azure specific stuff and automatically rule it out as a solution

[@mekowgli]

They require serious compute, so it's only fair to charge money for it. Although Azure pricing is, as usual, pretty wild. But the good thing is that, from what I understand, the API schemas are pretty much shared between different providers (including local ones) so you can use the same libraries to connect to different services.

[@mostrealtutu]

No

[@demarcorr]

i dont wanna and you cant make me.