Python Download Files From SharePoint Using Office365 Rest Package Part 1

[@AlejoVesga]

Im developer and I was trying all day because the library didn't work for me and I don't know what you do in the video but I followed it step by step and it worked for me, the best.

[@codingwithlu]

Glad you were able to get it to work.

[@simmonslucas]

I have never worked with SharePoint. Also, Microsoft has soo much deprecated tech and I was really feeling the knowledge debt. I had been heading in the many wrong directions. I can't tell you how much I needed this series!

[@codingwithlu]

Hopefully the videos are helping you out! I'm working on a course (free course) were I will be going over in more details working with Python and Office 365 services, like SharePoint, OneDrive, Word, Excel, Outlook, etc. Courses coming soon!

[@vmdcortes]

This was the best video related to this I have ever seen, the documentation was not enough but with your video everything was clear, thank you so much!!! It worked perfectly!

[@codingwithlu]

Glad it was helpful!

[@pythondeveloper4105]

Awesome !!! Most awaited video....... Great work Lu.....You are super talented Proud subscriber 😄

[@codingwithlu]

Thanks glad to have you as a subscriber. More videos to come!

[@thaleslessacosta6259]

Thanks for the awsome video and code, man ! I've been searching for something like this for some time.

[@codingwithlu]

Glad it helped you out

[@jndesign]

Hey Lu, thanks for this. Very helpful. Doing an enterprise version with Token or Cert would be great for those of us working in an enterprise environment.

[@codingwithlu]

Will do! Currently doing some testing with token and once my testing is completed I will make a video

[@PatrickPM8]

Very approachable and knowledge video. Thank you Lu!

[@IAmDublin]

Worked like a charm, thank you!

[@ProgramadorMetrindust]

17/09/2024 y me sigue funcionando tu video tutorial, muchísimas gracias

[@Golm1982]

Really awesome video to understand the process!!

[@codingwithlu]

Glad you liked it!

[@Suppankuppan]

Hey Lu, do you have downloading files from MFA authenticated Sharepoint using python scripts?

[@renerivera8115]

Awesome source for python office365. Thanks

[@codingwithlu]

Glad you like it

[@wesleytsang2887]

Thanks for the tutorial Lu, this is very helpful. I do have a question: Does this library support OAuth? If so, can you provide any pointers on integrating that with your demo?

[@VLeite777]

Hey Lu! Thanks for this content, helped me out a lot!

[@alexl1966]

Thx! You made my day🤘🤘. The damn original docu s**cks, you never want get a list of a sharepoint folder if you do it like the original docu suggests....

[@codingwithlu]

I'm glad my video was able to help!

[@13kriztian]

Awesome. It works for me. Thanks!

[@gregamos7987]

Thanks for the video, appreciate the detailed explanations. I tried to run this to download files from my organization's Sharepoint site, and got stopped by the access policy(" AADSTS53003: Access has been blocked by Conditional Access policies. The access policy does not allow token issuance.") Is there any way the script can be modified to get around this roadblock? Or do you have another video on this?

[@LuisGabrielFabres]

hi.. excellent video. Like several who post here, I am waiting for your tests with access through a token. Thanks for sharing.

[@tyronebiggums957]

We are using MFA and username/password option does not work. Is it possible to use the acquire_token_for_app() method and download files from sharepoint? All examples I see online seem to be broken.

[@codingwithlu]

if you are using MFA, then you have two options. 1) will be to connect via client id and secret id. I'm doing some testing over this, but I'm having some issues. 2) you can create a user account that has limited access and disable MFA for that account.

[@hasnainhaider8191]

@@codingwithlu waiting for that video :)

[@rashedanwar1630]

Hi Lu, wondering if you have any video/example on App ID / Secret based authentication instead of Username and Password.

[@jahelramirez3729]

Thank you very much for this video, it has been of great help.

[@codingwithlu]

Glad it helped!

[@dineshkumarmk8600]

Yow Lu vera level ya nee.!!!

[@cordeiro.thiago]

que maravilha, funcionou perfeitamente (pt-BR) amazing, it worked perfectly

[@Python_Griffin]

Sério? Estou tendo problema com autenticação. " Access has been blocked by Conditional Access policies. The access policy does not allow token issuance" Tem ideia do que pode ser?

[@jenniferklemisch585]

Hi - thanks for detailed walk throughs. Have you sorted out how to download by GUID id rather than relative url? I'm connected with client id and secrets. The only way I could get the file list to work is by using get by title.().get_items(). Then I was able to get the File objects with get_file_by_id using the GUID. However, now I can't get the actual file binary or read the file object. Three isn't a relative url in the document library list for each item.

[@poloperson4072]

Hey Lu, Could you update this (really useful) video to show using the more up to date method of authenticating (after registering your app) as MS have switched off Basic Authentication as of October 2022.

[@simmonslucas]

I think the git for Office365-REST-Python-Client has a readme section on this, check it out.

[@poloperson4072]

@@simmonslucas Yeah, I looked at that and didn't get very far. I need a bit of hand holding, and Lu's videos are awesomely simple to follow. So I'm hoping he can show how to do this in the same way as he did this video...... kzbin.info/door/BYUl2ztNB6ZsDmUBZZzaYw ??

[@user-pp1vi7wj1w]

Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access ''. I am this exception

[@rajeevpunna8109]

if we had the pdf file inside the SharePoint folder can we get the number of pages inside the pdf file using the office365 rest client module without downloading the file into local?

[@codingwithlu]

I'm not sure if you can access metadata without downloading PDF object. What may work in this case is to download into memory (don't save locally) and get metadata from file to determine page #

[@akhilpulimala556]

Thanks a lot man❤

[@codingwithlu]

You're welcome!

[@filipensilvag]

Hello bro, can I make the script copy a folder from O365 and always replace the folder/files in the local directory?? like a daily backup? what line of code for this?

[@codingwithlu]

You can use the code in from this video. If you use the correct arguments you can get all the files from SharePoint folder and download to locate folder. If you are using Windows you can use the Task Scheduler tool to call your script and have it schedule to run daily, weekly, etc.

[@david.lake92]

Nice video! How would you approach downloading files from a sharepoint public folder shared by other people? (links that don't need to authenticate)

[@codingwithlu]

Never tried that. I will do some testing and get back to you.

[@r2rtrade]

Hi Thanks for this video, Can we download, save and upload the file at the same time (from one SharePoint site to another SharePoint site.?

[@CristianRodriguez-zn3yb]

Hello! how are you? i am new to python, i am using it to download csv from sharepoint online and take them to sql server, we were able to achieve it using dataframe but it takes too long. Do you have any idea how to improve? or some method that should be used?

[@keifer7813]

It didnt work for me. I keep getting a 403 error which im guessing is due to the admins at my company

[@najam1923]

Very thorough description appreciate it a lot, however lu, i am having an issue where the script is not working on a remote ssh server, but is working on my local device. Any hints for this issue?

[@codingwithlu]

Do you get any errors on the remote server?

[@najam1923]

@@codingwithlu The script keeps on running aprox 15 minutes later, it returns a bunch of connection errors. my solution is a much simpler version than yours, where i only wish to retrieve lists and store them as dataframes from sharepoint.

[@najam1923]

@@codingwithlu The issue was resolved, tnx anyways

[@cdsiema3914]

This was very helpfull thanks

[@codingwithlu]

Glad it was helpful

[@maitri_shabdanshi]

instead of downloading file how can we read file contents , file should be pdf, excel or doc

[@herman4975]

Hi! Thanks for such informing video! Can we for example download whole folder with all files?

[@codingwithlu]

yes, I have a part 7 coming out tomorrow. Be on the look out for that video! it will show you how to download all the folders (subfolders) and files.

[@andrearocha2851]

HI there, Your videos are excellent, but I was left with a doubt. If I create another document library, called Invoices for example, it would be at the same level of Documents,Home, Pages. How could I access this library? thank you

[@lulzingrofl]

Is there actual documentation on this module or is it just the examples provided?

[@codingwithlu]

I'm assuming you are referring to the Office 365 package. You can checkout there github repo (github.com/vgrem/Office365-REST-Python-Client), but not much documentation. My videos are based on that package on my own research and testing on how to use package and incorporated another method to use the package in a much easier way.

[@ravivenkatatejamucharla1802]

Hi Lu, very nice explanation ! I have a request for you , could u pls let us know how to use client id and secret key for uploading a file using python

[@codingwithlu]

I have been doing some testing and have had some issues. Once I get this figured out I will make a video.

[@Python_Griffin]

Well, I don't know what I'm doing wrong, but I could not download the files from my sharepoint. I received a big message of error: "An error occurred while retrieving token from XML response: AADSTS53003: Access has been blocked by Conditional Access policies. The access policy does not allow token issuance." This is because i'm using my student email account? I only have this account with access to a sharepoint and would be amazing if i can automatize this process. Could you please help me Lu? OBS: I am currently learning about python, so if I did something wrong please sorry me.

[@codingwithlu]

So, normally we you get the "Access has been blocked by Conditional Access policies" error message, that means that the Admin has sent some policies in places that will provide you from connecting using API. Not sure what policies they have but you mentioned it's a school account, so you may not have any luck on getting the admin to help with policy. The work around in these cases is to create client id and client secret, but again not sure if that will work in your case due to being a school account.

[@Python_Griffin]

@@codingwithlu oh, I imagined that. Anyway, thank you Lu! Your videos are amazing! I will try with another Account out of this domain.

[@Python_Griffin]

@@codingwithlu you had mentioned the client Id and client secret. This is about the Microsoft Graph API? I’m currently studying this and I will try to implement that in my case.

[@hichamdioury9779]

Hi Lu, I got this error: SharePoint()._get_files_list(folder) get_files(FOLDER_NAME) File "C:\%\%\download.py", line 27, in get_files files_list = SharePoint()._get_files_list(folder)

[@Mythosis]

It would be great if you could expand this to download recursively through folder structures. And add output as files are downloaded.

[@codingwithlu]

I do have a video (Part 7) that will should you how to download all files and subfolders. kzbin.info/www/bejne/oaOxiaNmjap2fsk See if this helps

[@Yuya0518]

Great video super helpful! How would you go about disabling ssl certificates with Office 365 rest API?

[@Cordycep1]

Hello: anywaya not store login credential in a script. Anyway to havethe login screen popup when access Azure? thanks

[@yashinshaik2498]

Hi I am getting from the code File.open_binary(conn, relative_url/). Could you please help me?

[@codingwithlu]

So I noticed that one of the parameters for open_binary() is the url path and filename. What is the relative_url? If the path is not valid you will get an error.

[@JustinKemp1992]

@@codingwithlu Every file/folder has a relative_url in the meta data. What I did was looped through the meta data of all the files in a folder, adding each file to a list that I want to download, while also saving the Name, TimeLastModified, and ServerRelativeUrl fields. I then loop through the list of files that I saved, downloading each one by using the server relative URL provided.

[@codingwithlu]

@@JustinKemp1992 yeah I did something very similar as well. I created a function to get the list of files from folder and loop though it to download files

[@marksmith4710]

Hi Lu, this has helped me to download the files but when I download the Excel files, it says "Excel file cannot open the file 'abc.xlsx' because the file format or file extension is not valid." Can you please help on this?

[@lesleychingwena2611]

Hello Mark, I am getting the same error, did you manage to solve it???

[@os365]

@@lesleychingwena2611 me too

[@alexandercoster1237]

Hello, Great video, you are a good teacher. I am getting the error message when I run the download script: mismatch tag: line 25, column 71. Any ideas on what is causing this?

[@codingwithlu]

Hello, I'm glad you liked the video. If you can email me the full error message I can take a look. [e-mail]

[@vijayalakshmi8282]

Hi Lu, I saw all you're videos it really helpful a lot, but for me requirement is different which is i need to download all folders of folders and files at a time can you please help me out

[@RahulCloud-t3i]

getting error -------Exception has occurred: ValueError An error occurred while retrieving token from XML response: AADSTS90023: Invalid STS request. can anyone suggest me

[@royandre-davidkoupoh9015]

Great video. Plz, do you know why it does not work with companies SharePoint ?

[@codingwithlu]

It should work with corporate SharePoint, unless they have added some conditions policies setup. For example like MFA. What type of error ate you receiving? Also I will be making a video on connect via token, instead of email account. That maybe a better solution in your case

[@royandre-davidkoupoh9015]

@@codingwithlu File "C:\Users koupoh\Downloads\UPLOAD\env\download_files.py", line 43, in get_files(FOLDER_NAME) File "C:\Users koupoh\Downloads\UPLOAD\env\download_files.py", line 27, in get_files files_list = SharePoint()._get_files_list(folder) File "C:\Users koupoh\Downloads\UPLOAD\env\office365_api.py", line 29, in _get_files_list root_folder.expand(["Files", "Folders"]).get().execute_query() File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\client_object.py", line 52, in execute_query self.context.execute_query() File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\client_runtime_context.py", line 142, in execute_query self.pending_request().execute_query() File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\client_request.py", line 81, in execute_query response = self.execute_request_direct(request) File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\odata\odata_request.py", line 38, in execute_request_direct return super(ODataRequest, self).execute_request_direct(request) File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\client_request.py", line 93, in execute_request_direct self.context.authenticate_request(request) File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365\sharepoint\client_context.py", line 249, in authenticate_request self._auth_context.authenticate_request(request) File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\auth\authentication_context.py", line 89, in authenticate_request self._provider.authenticate_request(request) File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\auth\providers\saml_token_provider.py", line 77, in authenticate_request self.ensure_authentication_cookie() File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\auth\providers\saml_token_provider.py", line 84, in ensure_authentication_cookie self._cached_auth_cookies = self.get_authentication_cookie() File "C:\Users koupoh\Downloads\UPLOAD\env\lib\site-packages\office365 untime\auth\providers\saml_token_provider.py", line 102, in get_authentication_cookie logger.error(e.response.text) AttributeError: 'NoneType' object has no attribute 'text'

[@yorkmena]

Does it work for SSO with MFA enabled?

[@codingwithlu]

it will not work with MFA. I'm doing some testing using client id. Once I get the testing completed I will make a video.

[@Galva94a]

Is it normal that you have to type your password in clear code?

[@codingwithlu]

You are referring to the .env file. So .env files are normally exclude from git repo. It is design to only exist locally. Which means when deploying to production you will need to create a new .env file. Another option is to set these variables as system variables and only user with access can view and edit them. If there is still concern with have you credentials set in .env file, then the next option is to use access token. I'm working on this video and it will be out soon

[@Galva94a]

@@codingwithlu Thank you for the answer!

[@Galva94a]

@@codingwithlu Also, do you know if it is possible to download a file *which has been shared with me* and of which I am not the owner? I was able to download a file of mine, but not a file shared with me!

[@josemanuelmedinacano833]

Hi, @@codingwithlu! Any news on the access token video? Thanks in advance

[@JustinKemp1992]

Very nice video, this is amazing. I have my own working version that does not use classes. My question is, have you or anyone else had issues with downloading XLSM / macro enabled Excel files? It downloads the file, but when I try to open the file Excel says it is corrupted or the file format is not correct. I manually downloaded the file and everything checks out fine.

[@gorky_fr]

didn't try out yet, watching to close the missing parts of my code. My intention was too to download xlsm files with macro enabled files. If i recieve the same error will update here.

[@codingwithlu]

I have not tested that out, but I will do some testing and see if I can figure out what is happening.

[@marksmith4710]

Hi Justin, have you found any solution to download the XLSM or XLSX files without any errors? Please share the solution if you have found any!! Thanks in advance!

[@marksmith4710]

@@codingwithlu Hi Lu, have you tried downloading the XLSX / XLSM files and opening it without any errors using your code ?

[@JustinKemp1992]

@@marksmith4710 nope never found a solution. I just asked that the department do "save as" and change it to XLSX after they are done using the macro they have.

[@Naraharisettiraviteja]

how to validate if the connection is successful and I am using right credentials

[@codingwithlu]

Are you getting an error message when you try to connect?

[@Naraharisettiraviteja]

@@codingwithlu the response is empty string. Actually I am trying to consolidate meta data of entire tenant.

[@shubhamjoshi6608]

I want to access my company SharePoint which uses SSO to login. In this case will the .env fill require additional line of variable for authentication?

[@keifer7813]

Lol same. Did you manage to find a solution?

[@sandeepvlogz9535]

Did you got any solution ?

[@shawerahmed]

Hey Lu. Please we need one with Access Token Ceredentials. I can't find that anywhere.

[@alexzir]

Hi! Does it work with MFA authentication?

[@xsoru5074]

and if my sharepoint have MFA activated?

[@imotumbokanaka4157]

use app password

[@PANDURANG99]

where I will get sharepoint id password AND what does I will do if I get authentication error?

[@maitri_shabdanshi]

instead of downloading file how can we read file contents @codingwithlu please reply

[@romerapazaramos9879]

I'm getting this error : IndexError: list index out of range. Please what is this error due to, as I have checked the URL extension but without the correct ones.

[@romerapazaramos9879]

I saw the problem, it is because the account has double authentication, one that would be the password and the other through cell phone number, in the latter as aria to enter ?

[@LeonelSilva-j9g]

Hi Lu, I got this error: File "/remote/us01dwt1p051/fgduarte/p4_ws/33v2/tools/lib/python-3.7.0/office365/runtime/auth/providers/saml_token_provider.py", line 140, in _acquire_service_token_from_adfs assertion_node = dom.getElementsByTagNameNS("urn:oasis:names:tc:SAML:1.0:assertion", 'Assertion')[0].toxml() IndexError: list index out of range Any thoughts? Thank you

[@Plok26]

Do you have a tutorial on how to insert data into a Sharepoint list sir?

[@camilocorredor8713]

Can I upload files if i'm not the owner ot the site?

[@shreyasgowda2336]

Files are being downloaded, however it looks like they do not have any contents.

[@codingwithlu]

I'm assuming you are not getting an error message, just getting files with no data. In video there is a section where I have to specify the .content of the response. Can you confirm you are doing it that way as well?

[@shreyasgowda2336]

Thanks for the reply Lu. There are no errors, I am able to access the file names, however after they are saved to my local machine they become 1kb files with no content in them. Yes, I am using the .content of the response from the "download_files" function. (file.content) While checking the contents of file.content got an error "System.ArgumentException: Server relative urls must start with SPWeb.ServerRelativeUrl" Even the response I got for file is (400)

[@shreyasgowda2336]

Figured it out, my SharePoint link for the file starts with teams not sites. Made that change and am able download the files now. Again appreciate the time you take out to make these videos.

[@codingwithlu]

@@shreyasgowda2336 glad that my videos have helped

[@UpYours101]

Does this support SP2016?

[@codingwithlu]

Are you referring to On-Premise?

[@venkataramanapotluri1758]

root_folder.expand(["Files","Folders"]).get().execute_query() is throwing an error AttributeError: 'Folder' object has no attribute 'get' can please help if I am doing wrong

[@ВелиславаКрумова-ч4ж]

I have the same problem. Did you find a solution?

[@mewmew14344]

Help, i get this error An error occurred while retrieving token from XML response: AADSTS90023: Invalid STS request i'm downloading files from out company sharepoint.

[@mewmew14344]

i already have a client id and client secret but don't know how to apply to code.

[@Ss-rn4wn]

thanks a lot!! can you also do Upload lists and pull/push emails from sharepoint using office 365 too ..

[@simonmillan6524]

Hey really good content, it really helps. Can you please push the updates to github?

[@codingwithlu]

I have pushed to github. Check out github.com/iamlu-coding/python-sharepoint-office365-api

[@simonmillan6524]

@@codingwithlu oh thank you good sir. I was cloning another repo. Here, have a suscribe!! Thank you again!!

[@niranjanvelagapudi4273]

I'm getting this error : NoneType' object has no attribute 'text' and whatever I do I am getting the same error . Please respond , as I need to implement this in a chatbot project in my organization , Please respond quickly

[@Alloic]

amazing tutorial, thank you

[@srinathgb1321]

Hi Lu, I'm getting an error like this - assertion_node = dom.getElementsByTagNameNS ("urn:oasis:names:tc:SAML:1.0: assertion", 'Assertion')[0].toxml() IndexError: list index out of range

[@srinathgb1321]

Any idea on this ?

[@srinathgb1321]

env\env\lib\site-packages\office365 untime\auth\providers\saml_token_provider.py" in_acquire_service_token_from_adfs

[@codingwithlu]

So it seems you are using a different code base then my video. If you are using Javascript I'm not sure what the error is.

[@shanghua3350]

@@srinathgb1321 Did you find any solution on that error? I got the same error today.

[@yashinshaik2498]

Hi Lu, I am now getting this error . Initially my code was working. With Microsoft moving away from "Basic/Legacy" Authentication and towards Modern Authentication using OAuth. Could you please share solution for it?

[@ganeshawithyou7945]

Hi you are great! you saved me man. However, still facing a small issue. I have downloaded file but there is no valid content inside the file. instead error message written inside the file - {"error" :{"code":"-2147024809, System.ArgumentException","message":{"lang":"en-US","value":"Server relative urlsmust start with SPWeb.ServerRalativeUrl"}}} Can u pls help?

[@codingwithlu]

This is very weird. So it doesn't download the file, but the contents is the error message. I never seem this issue before. If you can email me your code (without credential) I can review to see if I see anything that stands out. My email is located in about section

[@manueltraverso1634]

it works perfectly for me, but when trying to use this method "conn.web.ensure_folder_path(target_folder_url)" got this error: "requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: " do you mind what it could be?

[@sebi_swars262]

As far as I know this approach doesnt work anymore... On the GitHub Page of the Office365-Rest Client package is a new Version