LangChain Sharepoint Loader

Efficiently Integrating and Managing SharePoint Data with Python: A Deep Dive into Custom Class Functions for Enhanced Document Handling
In the rapidly evolving world of integrating artificial intelligence into our workflows, having efficient and tailored tools is crucial. Recently, while exploring the capabilities of the LangChain library for document management, I encountered a significant hurdle: the lack of native functionality to interact with documents stored on SharePoint. After several hours of unsuccessful attempts to make LangChain work with SharePoint, I decided to develop my own solution. This video delves into the challenges faced with LangChain, the process of creating a custom solution, and why it is essential for developers and businesses to adapt AI tools to their existing infrastructures.
Link to my Medium.com article: / filling-the-gap-develo...
Link to my GitHub: github.com/ericvaillancourt/L...
Support my work by buying me a coffee or two! buymeacoffee.com/evaillancourt

[@pranavthirumalai126]

Thanks for the excellent video - one question: once the data is loaded, how do we feed/expose it to an llm? I'm using the AzureChatOpenAI client to instantiate my LLM

[@eric_vaillancourt]

You need to add the docs into a vector Database like this: db = Chroma.from_documents( docs, embedding=embeddings, persist_directory="emb" ) after all docs are saved, you can use this chain: from langchain.vectorstores.chroma import Chroma from langchain_core.output_parsers import StrOutputParser from langchain_core.prompts import ChatPromptTemplate from langchain_core.runnables import RunnablePassthrough from langchain_openai import AzureOpenAIEmbeddings from langchain_openai import AzureChatOpenAI import os os.environ["AZURE_OPENAI_API_KEY"] = "..." os.environ["AZURE_OPENAI_ENDPOINT"] = ".openai.azure.com/" os.environ["AZURE_OPENAI_API_VERSION"] = "2023-06-01-preview" os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"] = "chat" chat = AzureChatOpenAI( openai_api_version=os.environ["AZURE_OPENAI_API_VERSION"], azure_deployment=os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"], ) embeddings = AzureOpenAIEmbeddings( azure_deployment="", openai_api_version="2023-05-15", ) # Once you have your docs, you need the save them in a vector database. # Once it is persisted, you can use: db = Chroma( persist_directory="emb", embedding_function=embeddings ) # The you create a retriever: retriever = db.as_retriever() # The the template: template = """Use the following interesting fact to answer the user's question: {FACT} Here is the user's question: {QUESTION} """ # Creating an instance of ChatPromptTemplate from the defined template. prompt = ChatPromptTemplate.from_template(template) # and finally, the chain: retrieval_chain = ( {"FACT": retriever, "QUESTION": RunnablePassthrough()} | prompt | chat | StrOutputParser() ) # the question needs to be adjusted for your data result = retrieval_chain.invoke("Give me a fact about the French language?") print(result) Hope this helps!

[@youssefbaklouti5062]

Thanks a lot for this video, I'm having a trouble in the loader.load() function it generates a BadZipFile: File is not a zip file output.If you have any solution please let me know

[@eric_vaillancourt]

I have updated sharepoint. py with my last version and I added download_all_files_to_local_folder. py to demo file download to local folder. Let me know if that fixes it!

[@youssefbaklouti5062]

@@eric_vaillancourt Apparently I couldn't figure out how to write the resources in the .env file can you please provide me with an example. I followed the instructions on the article but it didn't work out.Thank you in advance.

[@eric_vaillancourt]

@@youssefbaklouti5062 You need to create the .env file in your project folder. Here is an example: TENANT_ID=aaaaaa-ahdjhdhdjd-a714-18c3225b6c1e CLIENT_ID=f;gksjkldfjgklfdg-fg-sd-fgsdf-gsd CLIENT_SECRET=kfdlfkahsdlkfhasdlhfalksdjfaZtQLpayj SITE_URL=your-tenant-name.sharepoint.com:/sites/your-site RESOURCE=graph.microsoft.com/ # only for demo 1 O365_CLIENT_ID=dsldkjkasdjfkjsdfksldjfklsdjf9i39-gr-g-d O365_CLIENT_SECRET=dkfkghlskdfjgklsjdfgjskdfjgskjgjsdfkgjksldf DOCUMENT_LIBRARY_ID=fklghslkdfghjksldfjgdjfgksjdfkgjkdfjgkjdfkgjkdjg_ You need to update all the variables with your own information. You will find all of the information in the app registration on the Azure portal. Also you need to update the SITE_URL to point to your SharePoint site. Le me know if this helps!

[@youssefbaklouti5062]

@@eric_vaillancourt Thank you very much that was so helpful, you're the best keep up ❤️❤️