Tools,

Donut on 100 receipts, tested

83%

of receipt fields exactly right

Get the script

Receipt photo in, JSON out

1.4 s a receipt, Mac GPU

806 MB model download

  1. Run pip install torch transformers pillow sentencepiece protobuf.
  2. Save donut.py below. Run python donut.py receipt.jpg.
  3. It prints the fields as JSON: items with count and price, subtotal, tax, total, cash and change.
donut.py
# pip install torch transformers pillow sentencepiece protobuf
# Run: python donut.py receipt.jpg
import json
import re
import sys

import torch
from PIL import Image
from transformers import DonutProcessor, VisionEncoderDecoderModel

M = "naver-clova-ix/donut-base-finetuned-cord-v2"
dev = "mps" if torch.backends.mps.is_available() else "cpu"
proc = DonutProcessor.from_pretrained(M)
model = VisionEncoderDecoderModel.from_pretrained(M).to(dev).eval()
tok = proc.tokenizer


def read_receipt(path):
    """Receipt photo to the CORD fields that Donut writes (a dict)."""
    px = proc(Image.open(path).convert("RGB"), return_tensors="pt").pixel_values
    task = tok("<s_cord-v2>", add_special_tokens=False, return_tensors="pt").input_ids
    with torch.no_grad():
        ids = model.generate(
            px.to(dev),
            decoder_input_ids=task.to(dev),
            max_length=model.decoder.config.max_position_embeddings,
            pad_token_id=tok.pad_token_id,
            eos_token_id=tok.eos_token_id,
            bad_words_ids=[[tok.unk_token_id]],
        )
    seq = proc.batch_decode(ids)[0].replace(tok.eos_token, "").replace(tok.pad_token, "")
    return proc.token2json(re.sub(r"<.*?>", "", seq, count=1).strip())  # drop the task tag


if __name__ == "__main__":
    print(json.dumps(read_receipt(sys.argv[1]), indent=1, ensure_ascii=False))

Three misreads on one receipt

A printed receipt with six items, a subtotal of 238,000, service 17,850, tax 25,585 and a total of 281,435
From CORD by NAVER Clova, CC BY 4.0. Cropped and resized. Donut read the full photo.

Donut wrote:

{"menu": [
  {"nm": "RedVelvet Cheese Slc", "cnt": "1", "price": "38,000"},
  {"nm": "Chicken Baked Rice", "cnt": "1", "price": "55,000"},
  {"nm": "Fish n Chips", "cnt": "1", "price": "65,000"},
  {"nm": "Mushroom Soup", "cnt": "1", "price": "35,000"},
  {"nm": "Chamoville", "cnt": "1", "price": "25,000"},
  {"nm": "Iced tea", "cnt": "1", "price": "20,000"}],
 "sub_total": {"subtotal_price": "238,000", "service_price": "17,650", "tax_price": "25,585"},
 "total": {"total_price": "281,435", "total_etc": "100,000", "cashprice": "200,000", "changeprice": "18,565"}}

87 of 95 totals right

Fields Donut wrote letter for letter, 100 receipts
Field Right Of
Total 87 95
Item prices 217 246
Item counts 199 220
Item names 194 251
All fields 1,079 1,301

Donut wrote 1,291 fields in all, so 84% of what it wrote was right. Names miss most: one wrong letter makes the whole name wrong.

More items kept with their price

Menu items read with their price, of 237
Donut, CORD model
81%
RapidOCR lines
70%
Our Actor
44%
Tesseract --psm 6
22%

An item counts when one output line has all its name words and its price. RapidOCR lines is the script of Receipt OCR in Python.

It skips what is not a field

Words found, of 2,373 on the receipts
Engine Found
RapidOCR lines 83%
Our Actor 68%
Donut, CORD model 67%

Donut writes no shop name, address or labels such as TOTAL. If you need the whole receipt as text, use an OCR engine.

donut-base is not an OCR engine

37% of words, best reading task

12 of 100 receipts empty, with each task

donut-base is the model before the receipt training. I asked it to read all the text with its two reading tasks, <s_iitcdip> and <s_synthdog>. Use the CORD model for receipts.

It can write items that are not there

On one receipt with 4 items, Donut wrote 8 extra items of loose letters first:

N O
N O
E T T E O
N O
E T E T E T E P O
E S O
E M A O
N O
T T AHU GORENG 28,000 1 O

In all, 18 of the 261 item names it wrote share no word with the receipt. That count includes the 8 above and misreads like Chamoville.

MIT model, CC BY 4.0 receipts

I read the licenses before the download. The four naver-clova-ix Donut models have the MIT tag, and the clovaai/donut code is MIT. CORD is CC BY 4.0, so credit it when you show its receipts.

Our Actor is for PDFs, not receipts

Our Actor reads the whole page as text. It found 68% of the words, but kept only 44% of items with their price. For receipt fields, use Donut or RapidOCR lines. For scanned PDFs, use the Actor.

How I tested it
  • Receipts: the 100 test receipts of CORD v2, 2,373 true words, 237 menu items with a name and a price.
  • Why the test split: the CORD model trained on CORD's train receipts. The receipts of Receipt OCR in Python are train receipts, so this page uses another set.
  • Fields: Donut's JSON and the true JSON, each cut into field and value pairs, compared letter for letter.
  • Words: true words found in any order, with the scorer of the receipt page.
  • naver-clova-ix/donut-base-finetuned-cord-v2 revision 8003d43, transformers 5.22.2, torch 2.15.0, on an Apple Silicon Mac GPU, at the lowest priority, one receipt at a time. One run.
  • RapidOCR 3.10.0 and Tesseract 5.5.4 on the CPU. The Actor row is build 0.1.65 code, run on this Mac with its default settings.
  • Time: the median per receipt, with the model loaded. Donut picks the most likely next token, no beam search.
What I did not test
  • Receipts from outside CORD, other countries and languages.
  • CPU speed, beam search and training on your own receipts.
  • The DocVQA and RVL-CDIP Donut models.
Sources

Two ways to skip the work

Run it yourself. Our Apify Actor turns scanned PDFs into Markdown, tables included.

Try the Actor

Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free.

Get one page free

Questions people ask

Can Donut read any receipt?

The CORD model reads the CORD fields: items, counts, prices, subtotal, tax and total. I tested it only on CORD's 100 test receipts. Receipts from other countries were not tested.

Is Donut better than OCR for receipts?

For items with prices, yes on these 100 receipts: 193 of 237 items, against 166 for RapidOCR lines. For all the text, no. It skips the shop name, the address and the labels.

What license does Donut have?

The naver-clova-ix Donut models have the MIT tag on Hugging Face, and the clovaai/donut code is MIT. The CORD receipts are CC BY 4.0.