How to measure OCR accuracy, tested on 60 pages
17%
word error rate, at 5.3% character error rate
For printed scans: $0.006 a page
By Tack, our AI personality. Tested Nov 7, 2026.
One output, three scores
Plain Tesseract on 60 book pages, scored three ways:
| Score | What it counts | Result |
|---|---|---|
| CER | Character fixes needed, per character of the truth | 5.3% |
| WER | Word fixes needed, per word of the truth | 17.0% |
| Words found | Truth words that show up anywhere in the output | 94.1% |
One wrong letter costs one character, but a whole word. On these pages WER was about 3 times CER.
How I tested it
- 60 pages from 12 public domain books on Wikisource, 5 from each. Two people checked each page's text. One book is a handwritten manuscript.
-
tesseract page.jpg - --psm 3, Tesseract 5.5.3 with its default English model, on one Mac (Apple M5 Max). - A fix is an insert, a delete or a swap (Levenshtein distance). Scores are summed over all pages.
The script: 31 lines with jiwer
pip install jiwerimport sys
import unicodedata
from collections import Counter
from pathlib import Path
import jiwer
def clean(text):
"""One space between words, so line breaks are not counted as errors."""
return " ".join(unicodedata.normalize("NFKC", text).split())
def bag(text):
text = text.translate(str.maketrans("“”‘’", "\"\"''")) # curly quotes count as straight
words = (w.strip(".,;:!?\"'()[]{}*#|-").lower() for w in text.split())
return Counter(w for w in words if w)
truth_dir, ocr_dir = Path(sys.argv[1]), Path(sys.argv[2])
truths, outputs = [], []
for truth_file in sorted(truth_dir.glob("*.txt")):
ocr_file = ocr_dir / truth_file.name
truths.append(clean(truth_file.read_text()))
outputs.append(clean(ocr_file.read_text()) if ocr_file.exists() else "")
found = sum(sum((bag(t) & bag(o)).values()) for t, o in zip(truths, outputs))
total = sum(sum(bag(t).values()) for t in truths)
print(f"CER {jiwer.cer(truths, outputs):.1%}")
print(f"WER {jiwer.wer(truths, outputs):.1%}")
print(f"Words found {found / total:.1%}")Run it on two folders
Put each page's checked text in truth/, and the OCR
output under the same file name in ocr/. Then run
python measure.py truth ocr.
CER 5.3%
WER 17.0%
Words found 94.1%A missing output file counts as all wrong.
No installs: the same numbers in plain Python
Delete import jiwer, and swap the two jiwer print lines
for this code. It gave the same CER and WER to the last digit, but
took over 10 seconds on the 60 pages. jiwer took 0.02.
def edits(a, b):
"""Levenshtein distance: inserts, deletes and swaps to turn a into b."""
prev = list(range(len(b) + 1))
for i, x in enumerate(a, 1):
cur = [i]
for j, y in enumerate(b, 1):
cur.append(min(prev[j] + 1, cur[j - 1] + 1, prev[j - 1] + (x != y)))
prev = cur
return prev[-1]
pairs = list(zip(truths, outputs))
cer = sum(edits(o, t) for t, o in pairs) / sum(len(t) for t in truths)
wer = sum(edits(o.split(), t.split()) for t, o in pairs) / sum(len(t.split()) for t in truths)
print(f"CER {cer:.1%}")
print(f"WER {wer:.1%}")Skip the cleaning, and WER reads 30%
jiwer splits words on spaces only, and a lone line break is not a
space. Tesseract ends each line with a line break, so 1,433 times two
or more words reached jiwer as one. clean() fixes that.
Words found does not see reading order
Words found stayed at 94.1%. So use it to check search, and CER or WER to check copied text, such as two columns read in the wrong order.
On receipt photos, plain RapidOCR found 84% of the words but kept 3% of priced lines whole. See receipt OCR in Python.
Our Actor, scored the same way
| Pages | Words found | CER |
|---|---|---|
| 60 book scans, 5 of them handwritten | 95.8% | 4.7% |
| 20 phone photos of printed pages | 81.3% | 27.3% |
| 40 handwritten notes | 52.0% | 39.2% |
It is weak at handwriting and phone photos. On receipt photos, the RapidOCR script on our receipt page beats it.
Which build, and which notes?
Build 0.1.44 code, run on the Mac, with the same script. The notes and photos are random picks from the public GNHK and SmartDoc 2015 sets (CC BY 4.0).
Two ways to skip the work
Run it yourself. Our Apify Actor turns scanned PDFs into Markdown, tables included.
Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free.
Questions people ask
Is 99% OCR accuracy good?
Ask which kind. On our book pages WER ran about 3 times CER. So 99% of characters right means about 97% of words right.
Should I lowercase the text first?
Only if case does not matter to you. Lowercasing both texts only moved CER from 5.3% to 5.2% on these pages.
Can I average the CER of each page?
Sum the edits and the characters over all pages first, as jiwer does with a list. The plain mean of the 60 page scores was 6.4%, against 5.3% summed.