Donut on 100 receipts, tested
83%
of receipt fields exactly right
Receipt photo in, JSON out
1.4 s a receipt, Mac GPU
806 MB model download
-
Run
pip install torch transformers pillow sentencepiece protobuf. -
Save donut.py below. Run
python donut.py receipt.jpg. - It prints the fields as JSON: items with count and price, subtotal, tax, total, cash and change.
donut.py
# pip install torch transformers pillow sentencepiece protobuf
# Run: python donut.py receipt.jpg
import json
import re
import sys
import torch
from PIL import Image
from transformers import DonutProcessor, VisionEncoderDecoderModel
M = "naver-clova-ix/donut-base-finetuned-cord-v2"
dev = "mps" if torch.backends.mps.is_available() else "cpu"
proc = DonutProcessor.from_pretrained(M)
model = VisionEncoderDecoderModel.from_pretrained(M).to(dev).eval()
tok = proc.tokenizer
def read_receipt(path):
"""Receipt photo to the CORD fields that Donut writes (a dict)."""
px = proc(Image.open(path).convert("RGB"), return_tensors="pt").pixel_values
task = tok("<s_cord-v2>", add_special_tokens=False, return_tensors="pt").input_ids
with torch.no_grad():
ids = model.generate(
px.to(dev),
decoder_input_ids=task.to(dev),
max_length=model.decoder.config.max_position_embeddings,
pad_token_id=tok.pad_token_id,
eos_token_id=tok.eos_token_id,
bad_words_ids=[[tok.unk_token_id]],
)
seq = proc.batch_decode(ids)[0].replace(tok.eos_token, "").replace(tok.pad_token, "")
return proc.token2json(re.sub(r"<.*?>", "", seq, count=1).strip()) # drop the task tag
if __name__ == "__main__":
print(json.dumps(read_receipt(sys.argv[1]), indent=1, ensure_ascii=False))
Three misreads on one receipt
Donut wrote:
{"menu": [
{"nm": "RedVelvet Cheese Slc", "cnt": "1", "price": "38,000"},
{"nm": "Chicken Baked Rice", "cnt": "1", "price": "55,000"},
{"nm": "Fish n Chips", "cnt": "1", "price": "65,000"},
{"nm": "Mushroom Soup", "cnt": "1", "price": "35,000"},
{"nm": "Chamoville", "cnt": "1", "price": "25,000"},
{"nm": "Iced tea", "cnt": "1", "price": "20,000"}],
"sub_total": {"subtotal_price": "238,000", "service_price": "17,650", "tax_price": "25,585"},
"total": {"total_price": "281,435", "total_etc": "100,000", "cashprice": "200,000", "changeprice": "18,565"}}
- Chamomile became Chamoville, and Iced Tea became Iced tea.
- Service 17,850 became 17,650. The total is right, so the lines no longer add up to it. Check the sum before you trust the fields.
87 of 95 totals right
| Field | Right | Of |
|---|---|---|
| Total | 87 | 95 |
| Item prices | 217 | 246 |
| Item counts | 199 | 220 |
| Item names | 194 | 251 |
| All fields | 1,079 | 1,301 |
Donut wrote 1,291 fields in all, so 84% of what it wrote was right. Names miss most: one wrong letter makes the whole name wrong.
More items kept with their price
An item counts when one output line has all its name words and its price. RapidOCR lines is the script of Receipt OCR in Python.
It skips what is not a field
| Engine | Found |
|---|---|
| RapidOCR lines | 83% |
| Our Actor | 68% |
| Donut, CORD model | 67% |
Donut writes no shop name, address or labels such as TOTAL. If you need the whole receipt as text, use an OCR engine.
donut-base is not an OCR engine
37% of words, best reading task
12 of 100 receipts empty, with each task
donut-base is the model before the receipt training. I asked it to
read all the text with its two reading tasks,
<s_iitcdip> and <s_synthdog>.
Use the CORD model for receipts.
It can write items that are not there
On one receipt with 4 items, Donut wrote 8 extra items of loose letters first:
N O
N O
E T T E O
N O
E T E T E T E P O
E S O
E M A O
N O
T T AHU GORENG 28,000 1 O
In all, 18 of the 261 item names it wrote share no word with the receipt. That count includes the 8 above and misreads like Chamoville.
MIT model, CC BY 4.0 receipts
I read the licenses before the download. The four naver-clova-ix Donut models have the MIT tag, and the clovaai/donut code is MIT. CORD is CC BY 4.0, so credit it when you show its receipts.
Our Actor is for PDFs, not receipts
Our Actor reads the whole page as text. It found 68% of the words, but kept only 44% of items with their price. For receipt fields, use Donut or RapidOCR lines. For scanned PDFs, use the Actor.
How I tested it
- Receipts: the 100 test receipts of CORD v2, 2,373 true words, 237 menu items with a name and a price.
- Why the test split: the CORD model trained on CORD's train receipts. The receipts of Receipt OCR in Python are train receipts, so this page uses another set.
- Fields: Donut's JSON and the true JSON, each cut into field and value pairs, compared letter for letter.
- Words: true words found in any order, with the scorer of the receipt page.
- naver-clova-ix/donut-base-finetuned-cord-v2 revision 8003d43, transformers 5.22.2, torch 2.15.0, on an Apple Silicon Mac GPU, at the lowest priority, one receipt at a time. One run.
- RapidOCR 3.10.0 and Tesseract 5.5.4 on the CPU. The Actor row is build 0.1.65 code, run on this Mac with its default settings.
- Time: the median per receipt, with the model loaded. Donut picks the most likely next token, no beam search.
What I did not test
- Receipts from outside CORD, other countries and languages.
- CPU speed, beam search and training on your own receipts.
- The DocVQA and RVL-CDIP Donut models.
Sources
Two ways to skip the work
Run it yourself. Our Apify Actor turns scanned PDFs into Markdown, tables included.
Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free.
Questions people ask
Can Donut read any receipt?
The CORD model reads the CORD fields: items, counts, prices, subtotal, tax and total. I tested it only on CORD's 100 test receipts. Receipts from other countries were not tested.
Is Donut better than OCR for receipts?
For items with prices, yes on these 100 receipts: 193 of 237 items, against 166 for RapidOCR lines. For all the text, no. It skips the shop name, the address and the labels.
What license does Donut have?
The naver-clova-ix Donut models have the MIT tag on Hugging Face, and the clovaai/donut code is MIT. The CORD receipts are CC BY 4.0.