[3] Importing financial statements from a webpage using Python

[@expat2010]

You know, I'm not much interested in financial statements and CPA stuff, - more data science and web scraping, but your videos are so good, that I'm learning more from them than others directly targeted to those areas.

[@PythonicAccountant]

Thanks! Very happy to hear!

[@lilyliu2174]

Practical tips cleaning and pre-processing the raw table! Good job!

[@PythonicAccountant]

Thanks! Just sharing what I like to do to make life easier :)

[@eurocanuck]

Finally videos for us bean counters! Good job so far. A couple things that may help with future videos (which you may have already done). 1. the volume of your videos are little low and make it hard to hear. Perhaps give the volume a boost in editing. 2. Would be great if you included link to sources you've used to make your videos. In this case the URL of the financial statements so that we can follow along and duplicate the results. Otherwise, I like what I see so far and look forward to progressing though the videos. Cheers!

[@PythonicAccountant]

Thanks so much for the feedback!!

[@nayanraj7587]

Thanks mate, this is awesome, super simple explanation provided !!!!

[@aliffdanial7924]

why do i get foribidden 403 error?

[@wunjo8586]

hey! nice video, simple and straightforward. Is there a way that we don't have to insert every time the html file for a specific company and instead just type the ticker symbol and we can download the data from it? I saw something with Edgar tools but they don't have all the rows. Thank you very much!

[@PythonicAccountant]

There should probably be a way that you could do this, I would have to go back and look so maybe I’ll add as a future video

[@jeremyruiz5032]

This is awesome! I tried to google importing XBRL financial statements using Python but I came up empty-handed. You said that you might make a video on XBRL in the future? It sounds like XBRL is faster as you don't have to guess which tables are the balance sheet or income statements like you do in XML? Anyways, thanks for the tutorial and happy holidays!

[@PythonicAccountant]

thanks for the note! you're right, there's not much out there, however I did stumble upon this code snippet which does seem to work generally for getting xbrl data... www.codeproject.com/Articles/1227765/Parsing-XBRL-with-Python - maybe I'll do a video on a version of that sometime soon! anything specific you'd be interested in doing with a set of public financials? Thanks, happy holidays to you also!

[@kelvinstones979]

I am so grateful to thechinahacks for helping me out .They got me a transfer of $37,00 to pay my debts .

[@antoinetremblay8523]

Hi! Excellent content and very useful. I wonder if you could do a video on how to import hundreds of historical financial statements from various companies in various but similar format (i.e. P&L/BS are don't have the same items, but there is similarities, as you very well know.). Is that doable?

[@PythonicAccountant]

It’s possible but what kind of companies? If publicly held then there are databases already out there you can use for free or cheap to that all that info. If non public data, what is the purpose? Just asking because the answer may dictate a different approach

[@antoinetremblay8523]

@@PythonicAccountant Thanks for replying 👌🏻 It’s for non public companies. Let’s say a large fund manager would like to aggregate all of its portfolio companies data into one database. Once that’s in place, it’s easier to track and compare performance. We’re taking 100-200 portfolio companies. All of them report and send their financial statements in different formats. Some Excel, some PDF, etc.

[@francisokoye5088]

Thanks a lot!Great Videos.Pls, can you make a video on how to extract the balance sheet, income statement, SOCE and CFS to a spreadsheet... which will be automated. Such that when you drop the FS in pdf in a folder, it automatically picks the FS item and the corresponding values to the appropriate years in Excel. Thanks

[@RizwanAhmedSurhio]

I think you can do it using various libraries in python such as Yfinance, pandas-DataReader etc. where you can call specific statement and specific caption and its value, and save it to excel.

[@davidezappa1983]

This video Is so so good, It helped me a lot

[@daniellevasconcellos633]

I need help lol. HTTPError: HTTP Error 403: - I’m on a Mac. pip 21.2.4 (python 3.9).

[@PythonicAccountant]

I probably need to redo this one! Some of the libraries may be out of date?

[@KhalilYasser]

Thanks a lot. Very cool.

[@saikiranreddy2664]

This is awesome! really helpful. I have a question though, the above process is for getting data of a single company. But if i want to get the data for a list of companies how can i do that ?

[@PythonicAccountant]

Easiest way would be turn this code into a function that takes a url and a table number as parameters and returns a dataframe of the table

[@kk_3354]

Great video sir! but I am wondering how you determine table 238 is the table, is there a way to dynamically figure it out? because I assume, say for Apple, Balance Sheet most likely won't be in Table 238.

[@sherlock27]

Thanks a lot man!

[@PythonicAccountant]

My pleasure!

[@MultiNarutoGamer]

Hello Pythonic Accountant, is this still the best way to get standardized balance sheets from all SEC 10k filings? I have tried to import the xml filings but it is hard to parse everything in a standardized layout.

[@perovicb]

great video, but sometimes its easier to just use blackbox, keep it up!

[@Goomba99-r9t]

I'm working on a similar project using requests.get and parsing through the info with bs4. I've been thinking about this.... couldn't you make it much simpler by importing the XML version for iXRBL filings? Might be an idea.

[@grzeszczukmaciej]

Is there easier way to replace "(" by "-" in pandas? Couldn't find explanation for lambda x online.

[@PythonicAccountant]

Maybe, but this is the easiest way for me. Try this for helping you understand lambda treyhunner.com/2018/09/stop-writing-lambda-expressions/

[@Michael-yu9ix]

When trying to import from the url in pandas, I get these error message: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076) urllib.error.URLError: Do you have any idea what is causing this?

[@PythonicAccountant]

Mi chael what versions do you have for Python, pip, and pandas? What OS are you on?

[@yordanpatronski1897]

is this an API?

[@PythonicAccountant]

Is what an API?

[@noorhashem7]

Is there a way to get htis information into excel??

[@PythonicAccountant]

Yep! Once you have it in a dataframe, which we do here, you just write one line of code to get it into excel: df.to_excel(‘filename.xlsx’)

[@noorhashem7]

@@PythonicAccountant Hmmm for some reason it did not work. I did df.to_excel("balanceSheer.xlsx") and it gave me an error that object has no attribute 'to_excel'

[@PythonicAccountant]

Noor Hashem if you want to send me the code you are trying to run and the full error message, I can take a look. [e-mail]

[@noorhashem7]

@@PythonicAccountant Hello, I have just emial you, thank you for your time!

[@noorhashem7]

@@PythonicAccountant I have sent a second email with a screenshot as the first one was not correct

[@unneeded02]

Please update, getting 403 error.

[@rachelwong949]

I did some googleing and found a way to get the same result by adding these lines of code: import requests, io header = {"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36", "X-Requested-With": "XMLHttpRequest"} r=requests.get(url, headers=header) Then I changed the read_html line to: df = pd.read_html(io.StringIO(r.text)) From there it all operated the same as in this video. Hope that's helpful

[@Sebasstian98]

Says lxml not found

[@ubaidurrehman8924]

Hello I need help please