Invoice Table Detection with Table Transformer

[@AI_Learner-v1l]

Hi Andrej, thank you very much for this video. I found an issue with long tables with Table-Transformer. If my pdf page has a long table that occupies whole page, then it is detected only partially, bottom part of table truncated and lost. Is there any solution to this?

[@AndrejBaranovskij]

Most likely this is table transformer limitation, it cant handle all table layouts. This is the reason I dont stick to single model in Sparrow, but also rely on Vision LLMs, such as Qwen2 for data extraction. See my latest videos where I describe it.

[@sudhirogale1687]

Hi Andrej, I have question, I am still struggling to understand how table structure works. As I understand AI model will let us know table location and OCR for text bounding boxes in that table. And then you will write custom logic to identify table structure - which may not be AI model but simple python function. is that true?

[@AndrejBaranovskij]

This was my attempt, but it doesnt work with more complex tables. If you check my latest videos, I focus on Vision LLMs usage in Sparrow, such as Qwen2 for data extraction, it works better and is more generic.

[@sudhirogale1687]

@@AndrejBaranovskij Thanks I will check.

[@TrườngNguyễnQuang-b9c]

I don't see the code file as shown in the Example section of the video in your github repo. Can you send me the "table_detector" code file as shown in the "Example" section of the video?

[@AndrejBaranovskij]

Code is inside Sparrow Parse lib. See table structure processor: github.com/katanaml/sparrow/tree/main/sparrow-data/parse/sparrow_parse/processors

[@mayssamayel6259]

what changes that u did when u implemented the code from the notebook

[@AndrejBaranovskij]

Different OCR processing, table transformer results cleanup. Merging column. Now working on fixing rows data. Quite a lot of changes.