Tools,

OpenCV OCR: text detection and recognition, tested

1,533 of 1,955

words found by OpenCV alone

Get the code

Tesseract still reads more words

10 scanned book pages, of 1,955 words
Run Words Time
OpenCV alone: EAST + CRNN 1,533 283 s
OpenCV boxes + Tesseract 1,865 83 s
Plain Tesseract 5.5.4 1,878 36 s

EAST and CRNN were made for scene text: a few words in a photo, like a shop sign. On these book pages, plain Tesseract was faster and read more. Times are rough: 2 threads on a shared Mac.

EAST finds the words, CRNN reads them

pip install opencv-python-headless

Then download two model files, frozen_east_text_detection.pb and text_recognition_CRNN_CH_2021sep.onnx (links under Sources). Save this as ocv.py next to them:

import sys

import cv2

cv2.setNumThreads(2)
det = cv2.dnn.TextDetectionModel_EAST("frozen_east_text_detection.pb")
det.setConfidenceThreshold(0.5).setNMSThreshold(0.4)
rec = cv2.dnn.TextRecognitionModel("text_recognition_CRNN_CH_2021sep.onnx")
rec.setDecodeType("CTC-greedy")
rec.setVocabulary(list("0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"))
rec.setInputParams(1 / 127.5, (100, 32), (127.5, 127.5, 127.5))

for path in sys.argv[1:]:
    img = cv2.imread(path)
    h, w = img.shape[:2]
    det.setInputParams(1.0, (w // 32 * 32, h // 32 * 32), (123.68, 116.78, 103.94), True)

Sort the boxes into lines

The rest of the page loop:

    boxes = sorted((cv2.boundingRect(b) for b in det.detect(img)[0]), key=lambda b: b[1] + b[3] / 2)
    lines, top = [], -h
    for x, y, bw, bh in boxes:
        if y + bh / 2 - top > bh / 2:
            lines.append([])
            top = y + bh / 2
        p = round(bh * 0.15)
        lines[-1].append((max(x - p, 0), max(y - p, 0), x + bw + p, y + bh + p))
    for line in lines:
        print(" ".join(rec.recognize(img[y0:y1, x0:x1]) for x0, y0, x1, y1 in sorted(line)))

Run python ocv.py page.jpg. Padding each box by 15% of its height found 1,533 words, not 1,392.

Tesseract does best on OpenCV's lines

PP-OCRv3, the OpenCV Zoo detector, boxes whole lines, and Tesseract reads a line well with --psm 7. Swap the detector and the reader, and drop the padding:

import pytesseract

det = cv2.dnn.TextDetectionModel_DB("text_detection_en_ppocrv3_2023may.onnx")
det.setBinaryThreshold(0.3).setPolygonThreshold(0.5).setUnclipRatio(2.0).setMaxCandidates(200)
det.setInputParams(1 / 255, (736, 736), (123.675, 116.28, 103.53), True)

        print(" ".join(pytesseract.image_to_string(c, config="--psm 7").strip() for c in crops))

On EAST's word boxes, Tesseract found 1,292 words. Whole lines work better.

The full ocv_tess.py
import sys

import cv2
import pytesseract

cv2.setNumThreads(2)
det = cv2.dnn.TextDetectionModel_DB("text_detection_en_ppocrv3_2023may.onnx")
det.setBinaryThreshold(0.3).setPolygonThreshold(0.5).setUnclipRatio(2.0).setMaxCandidates(200)
det.setInputParams(1 / 255, (736, 736), (123.675, 116.28, 103.53), True)

for path in sys.argv[1:]:
    img = cv2.imread(path)
    boxes = sorted((cv2.boundingRect(b) for b in det.detect(img)[0]), key=lambda b: b[1] + b[3] / 2)
    lines, top = [], -img.shape[0]
    for x, y, bw, bh in boxes:
        if y + bh / 2 - top > bh / 2:
            lines.append([])
            top = y + bh / 2
        lines[-1].append((max(x, 0), max(y, 0), x + bw, y + bh))
    for line in lines:
        crops = [img[y0:y1, x0:x1] for x0, y0, x1, y1 in sorted(line)]
        print(" ".join(pytesseract.image_to_string(c, config="--psm 7").strip() for c in crops))

EAST at 320x320 misses most words

The OpenCV tutorial gives EAST a 320x320 input. On a book page that shrinks the text to a few pixels: with Tesseract on each box, 105 of 1,955 words.

Give EAST the page size, cut down to a multiple of 32, as ocv.py does:

(w // 32 * 32, h // 32 * 32)

The zoo pair reads 3 words

The OpenCV Zoo pairs PP-OCRv3 with CRNN. CRNN squeezes each line box into 100 by 32 pixels, and its alphabet has no space.

The running head, The Boy Scouts of the Air, came out as Nerhig-honiogie.ds. CRNN_CH will not take a wider input, as (200, 32):

layers/reshape2_layer.cpp:179: error: (-215:Assertion failed) outTotal == inpTotal in function 'getOutShape'

The 36-character model loses capitals

EAST + CRNN, crops padded, of 1,955 words
Model Exact Case and punctuation ignored
CRNN_CH, 94 characters 1,533 1,679
CRNN_EN, 36 characters 1,240 1,544

My score counts a word only with the same case and punctuation. CRNN_EN outputs only digits and lowercase letters, so 599 of the words are out of its reach.

Errors, and mistakes with no error

Error Fix
Inconsistent shape for ConcatLayer Make the input size a multiple of 32.
Input size not specified Call setInputParams on the detector.
vocabulary is not specified Call setVocabulary with the model's characters.
Failed to parse ONNX model Check for a Git LFS pointer file. Get the real file.

No error, wrong result:

The full error lines

Each OpenCV line starts with OpenCV(5.0.0) and a long build path.

EAST at (1920, 3079):

layers/concat_layer.cpp:110: error: (-201:Incorrect size of input array) Inconsistent shape for ConcatLayer in function 'getMemoryShapes'

PP-OCRv3 at (1920, 3079):

layers/concat2_layer.cpp:127: error: (-215:Assertion failed) inpShape0[j] == inpShape_i[j] in function 'getOutShape'

No setInputParams:

model.cpp:115: error: (-201:Incorrect size of input array) Input size not specified in function 'processFrame'

No setVocabulary:

model.cpp:720: error: (-5:Bad argument) TextRecognitionModel: vocabulary is not specified in function 'decode'

No setDecodeType:

model.cpp:718: error: (-5:Bad argument) TextRecognitionModel: decodeType is not specified in function 'decode'

A Git LFS pointer file:

onnx/onnx_importer2.cpp:346: error: (-210:Unsupported format or combination of formats) Failed to parse ONNX model: lfs/text_detection_en_ppocrv3_2023may.onnx in function 'parseFile'
SystemError: <class 'cv2.dnn.TextDetectionModel_DB'> returned a result with an exception set
How I tested it
  • Pages: the 10 scans of OCR in n8n, proofread Wikisource book pages, 1,955 true words, as 1920 px color JPEGs.
  • OpenCV 5.0.0 (opencv-python-headless 5.0.0.93), pytesseract 0.3.13 and Tesseract 5.5.4, on an Apple Silicon Mac, on the CPU.
  • Settings from the OpenCV tutorial. EAST: confidence 0.5, NMS 0.4. PP-OCRv3: 736x736, as in the OpenCV Zoo. CRNN: 100x32.
  • One run at a time, lowest priority, 2 threads. Other jobs shared the Mac, so times are rough.
  • Score: true words found in order, with case and punctuation.
What I did not test
  • The tutorial's DB_IC15 and DB_TD500 models: their source, MhLiao/DB, has no license, so I did not run them.
  • Photos, signs and screens: I tested scanned book pages only.
  • A GPU, the C++ and JavaScript APIs, and the int8 models.
Sources

Scanned pages to text, no models

Run it yourself. Our Apify Actor reads PDF, PNG, JPEG, WEBP and TIFF for $0.006 a page, with no model files. Its code found 1,879 of these 1,955 words.

Try the Actor

Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free.

Get one page free

Questions people ask

Can OpenCV do OCR without Tesseract?

Yes. EAST finds the words and CRNN reads them, both in cv2.dnn. On 10 scanned book pages they found 1,533 of 1,955 words. Plain Tesseract found 1,878.

Why does EAST text detection miss most of my page?

The tutorial's 320x320 input shrinks a full page until the text is a few pixels tall. With Tesseract on each box that found 105 of 1,955 words. Give EAST the page size cut down to a multiple of 32: (w // 32 * 32, h // 32 * 32).

Why does TextRecognitionModel return lowercase text?

The model or the vocabulary has 36 characters: digits and a to z. For capitals and punctuation, use text_recognition_CRNN_CH_2021sep.onnx with its 94-character list.