[4] Use Python to extract accounting data from a PDF on the web

[@tomifg]

I wasted so much time with PyPDF2 and finally came across this video and pdfplumber. This was exactly what i needed. Thank you! I will definitely be back watching more of your videos

[@constituents07]

True!!

[@navsquid32]

What were your issues with PyPDF2?

[@tomifg]

@ after 4 years, I-thankfully-can't remember

[@June-c2q]

I'm also a CPA, and your clips are super useful. Thanks a lot.

[@navsquid32]

I’m not a CPA, and they are useful.

[@mshoaianh]

I have been binge watching your videos. Some steps I failed to get the same results...but appreciate your uploading!! this is unique on youtube

[@harshkantariya5362]

instead of iterating each time through rows, u can take the text of the page as variable and search with regular expressions. I think it should be faster and easier way to do if one needs more data from the file.

[@PythonicAccountant]

Very possible. I wasn’t as focused on optimizing the code, more just getting accurate outputs. But that makes sense as one way to improve performance! Thanks!

[@Lolpop751]

This worked great - PyPDF2 wasn't working and thought i was stuck! Thanks for the video!

[@tcbrj]

you saved my life, I was almost giving up from some project because it was impossible to get from pypdf2..... thanks! LIKED AND SUBSCRIBED

[@PythonicAccountant]

Sweet thank you!

[@lucatirel7301]

i was looking for some useful guide to convert pdf file to txt ordered ones for data minig and related tools and you have taught me more in 5 minutes that any other guide

[@PythonicAccountant]

Thanks, this is great to hear!

[@JamesHarrison008]

Just what i was looking for!

[@PythonicAccountant]

Great!

[@Sergio-pq3ri]

Perfect. Thank's bro, thumbs up

[@dddelgado05]

Which video would you recommend to watch to grab text inside the PDF table? Have a similar file but need text inside and struggling to figure out what I am missing. Very helpful videos thank you

[@inframan650]

Hello, very nice video. How can i extract data from pdf if the pdf is already downloadet on my computer?

[@PythonicAccountant]

Yes

[@DivyanshGeminiJIMS]

*How to get Biller's address & Sipper's address? Because there data comes in one line, how to differentiate them?* *Similarly for Code, Description, Qty, and Price.*

[@DivyanshGeminiJIMS]

Plz make a video on this, if possible🙏🏻🙏🏻🥺😶

[@muskangoyal484]

Did you do it? How can we differentiate them?

[@MohamedGamal-pj6wd]

Please I want to extract specific data from pdf and store them automatically in excel sheet how I can do that and thanks to much.

[@poojabanswal4623]

I want to do the same Did you find the way Please reply

[@Samarthkhandelwal09]

Hey! This is the first video I've watched by you. I am now interested in watching other videos Some video may tell me the purpose of using PDFplumber and other applications. I also have one query which is once I've got the code that gives right outputs can i run this code for extracting information from multiple PDF files directly into excel?

[@PythonicAccountant]

Thanks! Likely not unless you have PDFs in the same format. Otherwise you’d need to modify your code for each new format.

[@luizsenaluizsena]

You saved my live. No words to thank you.

[@PythonicAccountant]

You are welcome!

[@kamleshsay1903]

Hi..how can you help me with the regex toget the bill to and ship to address differentiate..please thankyou

[@DivyanshGeminiJIMS]

Did you found the solution for this? I have same issue in my project.

[@kamleshsay1903]

Yes..Try using bounding box method from pdfplumber library in python

[@yashpatel8632]

hello can we can extract data and directly fill from we have made with help of this code.

[@PythonicAccountant]

You’d likely need to customize to your PDF layout and output format, but feel free to use this code as a starting point!

[@ub9426]

Can you do from excel itself instead of pdf?

[@PythonicAccountant]

Yep super easy from excel! Just pd.read_excel()

[@vigneshvangala2235]

Hello, How do I get a next line of specific text.

[@PythonicAccountant]

Are you referring to this document or any document?

[@vigneshvangala2235]

@@PythonicAccountant Some other Document, I want to get text which is next line of the specific text. Can u please

[@camridgway3862]

Hey, while following along it was all good untill the balance part..keep getting name error balance not defined and no idea how to troubleshoot? Where is balance defined in the code above? Any help appreciated!

[@camridgway3862]

Ignore me i missed \ in (' ')

[@PythonicAccountant]

@@camridgway3862 I hate when I do that!!! :)

[@BradJ2485]

I'd love to see a Python tie-points video!

[@davidm3894]

Can you have a video on how to extract a report style pdf to excel? Meaning, let's say you have a report of invoices for many different companies and each invoice multiple purchases which have different SKUs. So the ideal way to export that to excel is to have the company name and invoice date repeat for each row that we have the unique SKU for that invoice (since the company name and date appear only once on an invoice but there are still multiple items purchased on the invoice). The final excel being a complete matrix of company, invoice date, and invoice detail.

[@PythonicAccountant]

David thanks for the suggestion. Definitely, I do this kind of extraction all the time! I’ll just have to find a close enough sample report to use, unless you know of one out there to use.

[@davidm3894]

@@PythonicAccountant I'll try to find one, or mock one up similar to what I am struggling with now! :)

[@PythonicAccountant]

David awesome, look forward to the challenge!

[@davidm3894]

@@PythonicAccountant How do I get the file to you?

[@PythonicAccountant]

David you can email it to [e-mail]

[@sathwikameenabad9789]

How can I extract street email or PO No from this pdf?

[@PythonicAccountant]

Same way, just use pattern matching to identify the line, split, and return the value

[@sathwikameenabad9789]

@@PythonicAccountant Can U please give me code for street email and PO no. and also printing bill to and ship to address separately,not in a single line ?

[@DivyanshGeminiJIMS]

​@@sathwikameenabad9789 Did you find the solution for this? I have same problem

[@beimberni6952]

Thanks for your vid, helped me to get my stuff done =)

[@ramonabreu258]

Hi there hope you are doing well. I am interested in building something like this using python: 1) user uploads a pdf invocie to a sharepoint 2) the system reads the pdf invoice 3) the system recognizes that it is a "gasoline invoice" becuase it is listed under the "gasoline invoice"folder 4) the system automatically books a journal entry debit gasoline expense and credit cash 5) everytime a new invoice is posted to the sharepoint the system automatically catches it and books it. Is something like this possible in python? I am willing to pay consultation and development fees related to this project. Regards

[@PythonicAccountant]

Hey there! Check out my reply to this same question on video 22. Thanks!

[@tiesnotesto]

Yes it is possible. I have done this for my work. Step 1 to 3 are straight forward. Step 4) depends on whether the accounting system you are using can accept instructions from python, in my case, I had to get pdf file information into an excel file using a template that the accounting system likes and then manually import the excel file into the accounting system to generate the journal entry.

[@dimpleklair7161]

Pls pls tell how to get sellers address and delivery address from an invoice.

[@PythonicAccountant]

You would want to use pattern matching, with regex. You could try using machine learning but that would be a bit more complex and might not be worth the effort

[@dimpleklair7161]

@@PythonicAccountant thank you so much for the reply

[@DivyanshGeminiJIMS]

@@dimpleklair7161 Did you found the solution for this? I have same issue in my project.

[@hannesbadenhorst8637]

Hi there , awesome tutoring.....how do I work this code for a local pdf file, on my pc, not from a url? I will be so happy if you can help

[@PythonicAccountant]

All you have to do is skip cells two, three, and four, and replace the invoice variable in cell five with the file name locally

[@hannesbadenhorst8637]

@@PythonicAccountant Awesome, thank you

[@angelav7999]

I downloaded my pdf invoice in anaconda environment and after i used the with pdfplumber.open("invoice.pdf") as pdf: page = pdf.pages[1] text = page.extract_text()

[@kissmysassafrass]

@@angelav7999 thank you!! i am a total newbie and could not get past this spot. high five for your help

[@kiranvanukuri9382]

And plz make a video on unstructured data like (.text) file with this file. And identifying exact names of related data ..plz make video on that sir

[@PythonicAccountant]

Do you have any example files that would work?

[@CodePursuit]

Thanks a lot !

[@PythonicAccountant]

You are welcome!

[@CodePursuit]

@@PythonicAccountant is there any way to extract address from the pdf ? Not a US based address but want to extract asian - household addresses from the pdf. The address may not exist as a key value pair

[@PythonicAccountant]

@@CodePursuit probably, if there’s a common pattern then you could write regex to capture it.

[@shivanijagani2492]

How will i extract billing and shipping address dynamically

[@PythonicAccountant]

You could use ChatGPT! See video 63…

[@shivanijagani2492]

can i make it for anymodel because i d not use openai for this as its paid,and chatgpt gave me regex method which i can not use as i do not know pdf,user will upload @@PythonicAccountant

[@celinesyriac6199]

How to extract if the document is already downloaded?

[@PythonicAccountant]

I cover that in future videos, but you can just open the local file using the location on your computer

[@simhz2221]

This looks very good and I'd like to try but I can't seem to be able to install pdfplumber through anaconda. I tried with "conda install -c gusdunn pdfplumber " but it gives me an error "PackagesNotFoundError: The following packages are not available from current channels : pdfplumber" Any idea why this is happening?

[@simhz2221]

Found the issue : conda is NOT supported even though it's documented on the anaconda page. To solve the issue, open the anaconda prompt and type pip install pip install pdfplumber

[@PythonicAccountant]

@@simhz2221 well done!

[@saurabhyadgire7282]

Can you provide similar video on reading content from txt file on the web

[@Qi2026]

Good stuff! I came across your blog and then went all the way to this channel. My question is, how can you extract multiple lines of this invoice? Say if I want invoice number and date? Thank you very much for producing these amazingly useful content :)

[@kiranvanukuri9382]

Nice sir super video

[@mkingopng]

hi, great videos. i'm following your tutorial 4 exactly, and i keep getting an error on cell 5 saying "AttributeError: module 'pdfplumber' has no attribute 'open'". any idea what i'm doing wrong? i've done the command line pip install of pdfplumber and everything seems fine. Got me stumped.

[@SteveMatyus]

make sure you didn't name your file pdfplumber.py ^_^

[@Ndofi]

Could add a video to explain do we extract data in multi-pdf file ?

[@PythonicAccountant]

Are you referring to pdf files that have multiple files embedded within one?

[@gulizotlu4877]

good job! Just I was wondering if that method is able to recognize hand writing ?

[@PythonicAccountant]

Thanks! Not this library as is, but you can use a trained machine learning model to recognize handwriting

[@walkwithus6536]

how to save it to csv?

[@PythonicAccountant]

If you have pulled it into a pandas data frame, you can just use the .to_csv method

[@my_opiniondemocracy6584]

how can I get the adress?

[@PythonicAccountant]

Just more pattern matching, make sure to know where in the document you are and grab those lines

[@sathwikameenabad9789]

Can we print the pdf exactly including whole text and borders ?

[@PythonicAccountant]

Not sure what you're asking. Any PDF reader can do that, print the PDF to your printer. Or display the full PDF on your screen.

[@sathwikameenabad9789]

@@PythonicAccountant displaying whole pdf including borders on screen using python

[@PythonicAccountant]

Sathwik Ameenabad you could use python to call a command prompt line to open the file in adobe reader. Is that what you mean? To automate opening the file for viewing? Otherwise I think you can also view the PDF pages using pdfplumber within the Jupyter notebook.

[@Traveltoexplore675]

Can anybody explain how this will benefit a company engaged in book keeping?

[@PythonicAccountant]

Are you asking about bookkeeping uses from this specific video about extracting data from a PDF, or about using python in general?

[@Traveltoexplore675]

@@PythonicAccountant about bookeeping uses from this?

[@PythonicAccountant]

@@Traveltoexplore675 bookkeeping uses for this could be things like turning anything that is in a PDF format into an Excel file that you need to perform some kind of calculation or record a journal entry or process an invoice or do a reconciliation, etc. If you don’t ever get anything in PDF format then this would not be very helpful

[@Traveltoexplore675]

@@PythonicAccountant thank you so much ..

[@simplethings6489]

Hi, I need to extract all the data from pdf and need to save in excel. But if pdf is having tables and images and semi structured pdf also it's not working. Any idea please. If you help it would be appreciated

[@PythonicAccountant]

Please note my code won’t work as a copy and paste, but can be used as a foundation for writing custom code for your specific PDF. If you are having trouble getting it to work, you can either 1) buy some proprietary PDF extraction software to do the trick, or 2) hire someone with more python experience to help code the PDF extraction

[@davidsanchezpamplona1264]

Do you know any method to delete vertical letter margin left line in the invoice with legal information? This line destroy the text in the rest of invoice

[@PythonicAccountant]

Hi, can you clarify what you mean by that? Or send an example?

[@davidsanchezpamplona1264]

@@PythonicAccountant There is an example in this link of we transfer: we.tl/t-IXV98CcfKN I have problems with vertical text in margin left. When i make extract_text() appears wrong. Thx

[@davidsanchezpamplona1264]

It is possible delete this part of the page with crop method.

[@DivyanshGeminiJIMS]

@@PythonicAccountant He is saying that, text is extracting linewise, he wants text columnwise. B'coz for example Shipper's address and Biller's address are coming in same line.

[@Hana2Ahmed]

Can you add the code below the video becoace it dosn't clear,if you don't mind

[@PythonicAccountant]

you can see the code here github.com/danshorstein/pythonic-accountant

[@python360]

@@PythonicAccountant Excellent video - please keep making them - you should write a book..seriously!

[@vallepusaiteja2768]

How to extract data from description column and notice column from pdf

[@Geeliowl]

Nice video, though when I tried to open pdf file with Pdfplumber, all the separator between numbers (, and .) being replaced by space. But look at your video, it works fine. Wonder why.

[@PythonicAccountant]

The comma and closed parentheses need to be replaced with an empty string, not a space. Open parentheses are replaced by a minus symbol. Don’t do anything with the period unless it’s not being used as a decimal.

[@letsdoitwithridhi8959]

ths code not working please help , at 3.46 time stamp, it is not wroking

[@PythonicAccountant]

What’s the error message?

[@izzyanalytics4145]

Exactly what I needed. Thanks!

[@sreedathps7368]

Hi bro, what if it's balance sheet and there are like 500 different templates for the balance sheet and I have to get the numbers from a particular column!?

[@PythonicAccountant]

Certainly possible if there is some structure you can use pattern matching on

[@sreedathps7368]

@@PythonicAccountant can I mail you regarding this? Because I am not able to completely sort it out. Can you please help me out?

[@PythonicAccountant]

sreedath ps sure [e-mail]

[@sreedathps7368]

@@PythonicAccountant Thank you bro I've send you a mail. Please help me out.

[@trackstar127]

How come when i try to use the same code i get a memory leak error? im not sure how to fix that, this is all new to me.

[@PythonicAccountant]

What’s the error say exactly? Also, what OS and python version are you using?

[@trackstar127]

@@PythonicAccountant I just downloaded it today so it should be the latest version (i believe im on 4.9.2) my os is windows 10. under ~\anaconda3\lib\site-packages equests\api.py in get(url, params, **kwargs) it says "# cases, and look like a memory leak in others." Then further down it goes on to say get the appropriate adapter to use , start time (approximately) of the request, and "nothing matches :-/". Invalid Schema i used the exact same syntax as you and the same invoice pdf link ( i took from searching that company).

[@trackstar127]

@@PythonicAccountant so looks like its working now, i think it may have had to do with my java path not being set in the environment variable.

[@PythonicAccountant]

@@trackstar127 glad it’s working now!

[@jasons.estrada8086]

great video

[@PythonicAccountant]

Jason Estrada thank you!

[@cuicuili7647]

AttributeError: module 'pdfplumber' has no attribute 'open'. who can help me solve this problem in cell 5????????

[@helomidnight8551]

I followed the steps one by one, but I got the No module named ‘pdfplumber’ error Has anybody any idea how can I fix this?

[@PythonicAccountant]

Hi, you have to install pdfplumber as it’s a third party library. Can typically be done using pip install from the command line.

[@helomidnight8551]

@@PythonicAccountant Thank you 🙂

[@filipzaezny4366]

Wow, seems so easy :)

[@PythonicAccountant]

Yes, exactly my thoughts =)

[@ubaidurrehman8924]

Hello I need help please

[@mdelbiondo]

What are you CPA auditors using this for in fieldwork? Create a macro to run this on 1000's of invoices in a search for AP? Excel nerd here who audits local governments and non-profits, and is trying to understand who to apply Python to everday auditing.

[@PythonicAccountant]

Create a python script to read in the entire audit client’s general ledger, perform reconciliation to trial balance, use to visualize transactions for unusual activity, perform disbursement / journal entry sampling; could also read in sub ledger details and reconcile to gl details. Automate trend analyses and roll forward each year. Read in 400 page pdf reports and foot them, load into excel, make much easier to audit. Just a few examples

[@Ndofi]

thanks very much for this video.

[@jgwang7968]

Hello, I am trying to extract date info from a PDF, which is in the middle of a row, how to do that? Thanks.