RAG with Azure AI Search and Azure Open AI in 9 minutes

[@davemcshane5292]

brilliant video!! Much better than the marketing ones from microsoft :)

[@ambarishg]

Wow, thanks!

[@UmairMateenKhan]

Awesome tutorial Ambarish. Keep up the good work.

[@limjuroy7078]

Here is my little 2 cents', can you do an end-to-end tutorial on how you do this, including creating Azure AI Service, Azure Storage as well as the coding part? 😅😅😅

[@ambarishg]

Great suggestion!

[@xyz-vv5tg]

Please. I need it. I have no idea on Azure or any of its services. I'm just exploring and getting confused

[@JohnPatrickGallagher]

I’ll second this, I’m getting Started and it’s a great tutorial so far!

[@samosertogo]

Well done! Brilliant work and very replicable. Thanks a lot!

[@ambarishg]

Many thanks!

[@paullopez_ai]

Great video! Thanks for sharing the code too, greatly appreciated.

[@ambarishg]

Glad to help!

[@VivekGupta-zc1ue]

Awesome content. Here is the question -- can we upload the confluence page url and get the results from same origin.

[@Momentum_Option_Buyer]

1:01 Is Azure search is or acts like a vector database that keeps the knowledge? Please clarify I have seen several videos but confused on this question.

[@KenGarff-zp1mh]

Thanks for the video!

[@murtuza.chawala]

Awesome Video I tried it too wanted to know any way we can do reinforcement learning making the bot train on our feedback

[@mohammadsohail6534]

Hi @ambarishg Great video Will the Azure Search AI store the indexed data or it is stored in StorageAcount

[@ambarishg]

In Azure AI Search

[@saideepj6998]

Thanks for such a detailed content. Is the uploaded data going to private ? will that data be trained with open source LLM's . How to add extra layer of security to make sure that the uploaded data is within my private GPT

[@Melophile_Gaurav]

Hey, this was helpful! :)

[@shashanksagar4878]

Do we not require any embeddings and vector database here? Also, is vector search not required? If so when is it required? Thank you

[@andonii46]

Amazing video! One question, if you have the data source a blob storage, if you delete a pdf file from that blob storage, when you run the indexer is the reference and information about this deleted document deleted from the index as well? thanks!

[@ambarishg]

The data is stored in the Azure AI Search index. Please note that if we have to remove a document from the index, we have to delete it from the index. Please also note that in the file upload_docs.ipynb, the function remove_from_index can remove documents from the index

[@akankshamakkar8538]

So here we need 3 services, right? azure openai, azure ai search and azure blob storage?

[@italoaguiar]

thanks! very informative video!

[@ambarishg]

Glad it was helpful!

[@limjuroy7078]

What if I want the app "also" has the ability to upload our own PDF file and query the content of the just uploaded file through UI?

[@ambarishg]

Yes it is possble

[@pooblock4092]

Does this work with files with size large than 16mb?

[@indrayne1840]

Sir I have one doubt, like we are using Azure Cognitive search and the index updates everyday midnight. So we have both new content and old content with us. Now what I want is to retrieve the most recent content first and than old content. How can I implement it?

[@pedromoraes5197]

Great video, Man!! Thanks for sharing.

[@ambarishg]

Thanks for watching!

[@ankan54]

have you tried creating a vector index in ACS? will vector search be better than semantic search?

[@ambarishg]

Good suggestion will try. Usually Hybrid search is better

[@ambarishg]

@ankan54 . Please have a look at the video kzbin.info/www/bejne/p3vPZHyaeLCZrKc where we discuss the following RAG with Azure AI Search with Azure OpenAI with different search techniques and Langchain with Conversation Chain, Prompt Template and Conversation Buffer 📍 Vector Search 📍 Hybrid Search 📍 Exhaustive KNN Search 📍 Hybrid Search with Semantic reranking 📍 RAG with Azure AI Search with Azure OpenAI 📍 Use Langchain with Conversation Chain 📍 Use Langchain with Prompt Template 📍 Use Langchain with Conversation Buffer

[@Momentum_Option_Buyer]

You didn't use vector database in this specific video (use case) to check data into vectors?

[@ambarishg]

Please watch the following for VECTORS 1. kzbin.info/www/bejne/eICzh5tjZbOAbrc - Vector , Hybrid, Semantic search with Azure AI Search in 11 minutes 2. kzbin.info/www/bejne/p3vPZHyaeLCZrKc -RAG With Azure AI Search using Vector/ Hybrid / Exhaustive KNN / Semantic Reranker in 15 minutes

[@sarak8467]

Hi Ambarish, thank you for the very helpful video! I have an enquiry and I hope you could help me with it. So, on the Azure Platform, I have created a a Search Service and an OpenAI Service and connected both in order to be able to ask questions about the data stored on an index. The data is in a Blob Storage container that has multiple folders in it. What I want to do when searching for information, is to filter the folders; I want to choose the folder (either by name or path), so that the search service only returns the content of that folder, and then the openAI service uses that content as a reference to generate an answer to my question. Do you know how I could do that?

[@ambarishg]

In the code we have added metadata while putting data into the search index. Please add your customized metadata while putting the data into the search index. While doing the Search on Azure AI , we should provide the filter with the proper metadata and it should filter accordingly. Hope this helps. May be I should so a followup video explaining this in detail. Thanks again for your questions.

[@sarak8467]

@@ambarishg Thank you so muh for your reply! IT would be perfect if you could show us how to do it. If you can, please show us how to do it on Azure and through code. (I use either Python or C#) Thanks again!

[@krishnavasu3499]

Great video! Very helpful. Instead of azure AI search, if I wanted my retriever to be Qdrant, would it be possible? I have a few files in Qdrant and would like to use Azure open AI to get the output from Qdrant and give the result to the user. If possible, how could I do it?

[@ambarishg]

Yes you can! kzbin.info/www/bejne/Y2aWqYyIa96Ed7s This shows RAG with QDRANT [ RAG with Opensource Tools - Qdrant and Mistral in 6 minutes ]. You may please modify this for Azure Open AI

[@ambarishg]

kzbin.info/www/bejne/nmWpYWycjM2snsk This shows RAG with LlamaIndex - Qdrant and Azure OpenAI in 9 minutes

[@abhinavanand6660]

hey in the section of env where could i find the index name of the ai search

[@ambarishg]

index = "" This is in github.com/ambarishg/AZURE-AI-SEARCH/blob/main/.env.sample

[@user-pm1lh3kr9k]

hey can you please put sample env file and explain the parameters

[@ambarishg]

Sure.I will add the ENV File

[@ambarishg]

Added the .env.sample file in the repository.Hope this helps