Receipt OCR Benchmark with LLMs

Extracting information from receipts is crucial for companies since hundreds of thousands of workers are submitting their work associated bills through receipts.

With the most recent developments in generative AI and large language models, information extraction accuracy has reached roughly human ranges.

Benchmark outcomes

We used Claude 3.5 Sonnet to measure the receipt information extraction accuracy of LLMs:

Dataset

We divided our dataset into two components:

• Top quality: Scanned, excessive decision receipts. These photos are aligned nicely, with excessive distinction.

• Low high quality: Photographed, low high quality receipts. These photos will not be aligned correctly, with no pre-processing to make distinction increased.

Our intention is to cowl real-life instances as a lot as doable.

We requested for a JSON output to make analysis simpler. Our immediate is: Please output the textual content on the PDFs in a correct JSON format.

Methodology

Outcomes have been evaluated at key-value pair degree:

• If a discipline contains the right label and worth, it’s marked as appropriate.

• If there are any character variations vs the bottom reality within the label or the worth, that row is marked as false.

Extraction accuracy: Variety of appropriately extracted key-value pairs divided by the entire variety of key-value pairs.

Subsequent steps

We’ll add extra LLMs (ChatGPT and so forth.) to this benchmark to look at their skill to information extraction higher.

What’s receipt OCR?

Receipt OCR (Optical Character Recognition) is a know-how that extracts information from scanned and digital receipts utilizing synthetic intelligence and machine studying algorithms. Receipt OCR parses the info, converts it to a structured format and captures particulars within the receipt, like date, objects, and costs.

To extend the accuracy of the OCR, the pictures ought to be:

• In increased decision

• Aligned nicely

• Freed from printing errors

You ought to be conscious of:

A lot of the receipt OCR instruments fail in matching the right merchandise with appropriate worth when there’s a word concerning the merchandise within the subsequent line with no pricing listed. In that case, it is not uncommon for instruments to learn the subsequent merchandise’s worth because the word’s worth. To see clearly, let’s have a look at the instance:

In such instances, the output of OCR might match “SpcyDlx +PJ” with the value 0.40, which isn’t appropriate. It’s doable particularly within the instances the place picture decision and high quality is low, and the picture shouldn’t be aligned straight.

We seen that within the case of low decision or printing errors (ink doesn’t cowl the letter utterly and so forth.), instruments are having bother in absolutely figuring out comparable letters and numbers. Like “8” and “9” or “5” and “6”. Additionally having bother in figuring out “/” and “1” is a typical case, particularly in dates.

• Receipt quantity

• Date

• Vendor identify

• Subtotal quantity

• Tax quantity

• Whole quantity

• Bought objects

• Receipt scanning: Scanning the receipt with excessive decision. OCR receipt scanning helps getting extra top quality photos than taking images of the receipts.

• Receipt processing: To extend distinction and readability of the enter picture, processing receipts could also be wanted.

• Receipt parsing: Parsing the receipt picture is crucial to investigate and seize information, it breaks down information into extra organized parts.

• Utilizing structured information: Structured information can be utilized to automate information entry in present programs like accounting software program. Related information can be utilized in so many instances like following the transaction date in monetary information and expense administration. By robotically extract information from receipts through the use of LLMs or receipt OCR apis can cut back errors and guide entry and will increase total effectivity with excessive accuracy.

FAQ

Exterior Hyperlinks