How To Integrate OpenAI With Azure Vector Search aka Azure Cognitive Search

[@shweta-lodha]

Code snippets and bit of explanation shweta-lodha.medium.com/step-by-step-tutorial-to-integrate-openai-with-azure-cognitive-search-vector-search-4d75b400675c

[@jorgeanicama8625]

Very straight to the point. Just the video i needed to connect many isolated concepts! Thank you very much. !

[@shweta-lodha]

Glad you find it useful. Stay tuned for more such bites 👍

[@soubarnobanerjee8257]

Just loving your contents! There are a lot of grey areas onto the LLMs and how they work if we integrate Azure services, especially the vector DB part. Using FAISS, Chroma, etc are good for POCs, that too performed internally.... but something which gets accepted in enterprise level is the one which you are talking about in this video!! Thank you

[@shweta-lodha]

Thank you 😊

[@deepikahari3790]

I could not find "generate_embeddings" function definition.

[@joannezhu101]

Thank you Shweta, really helpful for me to understand how the Cognitive search works. I have another question, wouldn't it be easier to use Azure openAI instead of OpenAI as the chat language model towards the end? I'll duplicate the code on my end. Thank you so much for sharing this with us.

[@KushwanthK]

thanks this is really good where is generate_embeddings function located and which embedding model are you using? share the notebook?

[@nikithajadhav-d2r]

Hi Thank you for the video ,i am getting httpresponseerror invalid parameter can u help me with this?

[@jatinnandwani6678]

Thanks Shwetha

[@MrRugev]

Thank u!!!!, your videos have helped me a lot.

[@shweta-lodha]

I'm so glad!

[@abhayrajput9546]

What is generate_embeddings ?, not there in your blog

[@shweta-lodha]

This implementation varies based on your choice - whether you are going with OpenAI embedding, hugging face or any other. Purpose of this function is to generate the embedding for the given text

[@Haraker21]

Hi, how would you suggest to implement this solution with multiple documents?

[@shweta-lodha]

It doesn’t actually matter from DB side, whether you are taking single or multiple docs. It only differs at a point where you are reading your docs. If you have multiple docs, you can load using some directory loader

[@OleksandrAkimenko]

This line is not fully displayed in the video could you kindly provide the rest of the line, please? SearchField(name="embedding", type=SearchFieldDataType.Collection (SearchFieldDataType.Single), searchable=True, vector_search) Thanks

[@OleksandrAkimenko]

as well as generate_embedding function

[@shweta-lodha]

I don't have the complete code, but I hope this snippet would work: fields = [ SimpleField(name="documentId", type=SearchFieldDataType.String, filterable=True, sortable=True, key=True), SearchableField(name="content", type=SearchFieldDataType.String), SearchField(name="embedding", type=SearchFieldDataType.Collection(SearchFieldDataType.Single), searchable=True, vector_search_dimensions = 1536, vector_search_configuration ="my-vector-config") ] index = SearchIndex( name=index_name, fields=fields, vector_search=vector_search )

[@shweta-lodha]

response = openai.Embedding.create( input="text", engine="text-embedding-ada-002") embeddings = response['data'][0]['embedding']

[@OleksandrAkimenko]

Thanks!@@shweta-lodha

[@joellim1]

Hi Shweta, your content is very clear and useful! How might we get the full .ipynb source code so we can try out the solution ourselves? For instance, generate_embeddings function is not shown.Thank you for all that you do!

[@shweta-lodha]

It is there on my Medium blog. Medium link in present on my profile page

[@joellim1]

Many thanks@@shweta-lodha

[@permanazainal6003]

@@shweta-lodha medium member only

[@dheerajsachdeva5735]

Hi! Can you please share the notebook that you used in the video? The medium site is for members only.

[@shweta-lodha]

That is the only option I have today. Sorry for that

[@hrishabhgupta7568]

I am following as per the tutorial but getting this error while creating the Index: vector_search_configuration is not a known attribute of class and will be ignored

[@shweta-lodha]

Are you sure, it’s an error and not a warning?

[@Shaikshavali.D]

Hi @shweta lodha, can we rename the index name in azure cognitive search.

[@shweta-lodha]

You can’t

[@OleksandrAkimenko]

This is so good, thank you so much

[@shweta-lodha]

Glad to know that 🥰

[@pavan1006]

In production, how do you deploy this python code ?

[@shweta-lodha]

You can create a web app and do it

[@HariTummuri]

can we use pdf files in the sameway?

[@shweta-lodha]

No, reader would be different

[@HemantKrSahu]

Thanks 🙏🏼 to you

[@shweta-lodha]

Always welcome

[@ssss-u7w]

thanks for the video.

[@shweta-lodha]

You're welcome