Remove lines before OCR: inpaint, do not paint white
0.6%
words wrong, lines through letters, inpainted
For printed scans: $0.006 a page
The code: mask, then inpaint
Find each long run of ink, widen it a pixel, and let
cv2.inpaint fill it.
pip install opencv-python-headless pytesseract
brew install tesseract
import cv2
import numpy as np
import pytesseract
def remove_lines(gray, length=60, grow=3):
"""Inpaint straight runs of ink at least `length` px long."""
ink = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
across = cv2.morphologyEx(ink, cv2.MORPH_OPEN, np.ones((1, length), np.uint8))
down = cv2.morphologyEx(ink, cv2.MORPH_OPEN, np.ones((length, 1), np.uint8))
mask = cv2.dilate(across | down, np.ones((grow, grow), np.uint8)) # cover soft edges
return cv2.inpaint(gray, mask, 3, cv2.INPAINT_TELEA)
gray = cv2.imread("scan.jpg", cv2.IMREAD_GRAYSCALE)
clean = remove_lines(gray)
text = pytesseract.image_to_string(clean, config="--dpi 300")
Set --dpi to your scan's real dpi. A NumPy image has
none.
How I tested it
- 24 pages made with Pillow at 300 dpi, so I know every word.
- 22 lines a page, each 6 random dictionary words and a number, in 4 fonts.
- Under each line of text, a 3 px line. On half the pages it touches the letters. On the other half it sits 14 px under the baseline.
- Then a scan look: a small tilt, blur, noise and JPEG.
- Tesseract 5.5.3 at psm 3, OpenCV 4.12, one Mac. Words wrong is WER, as on how to measure OCR accuracy.
- The code above, run as printed, gave the same result: 12 of 1,848 words wrong.
White paint cuts the letters
White paint takes the tails of g, p and y with the line. Tesseract read "geckotian gip lingual" as "aeckotian aip linaual". Inpaint draws the tails back.
Which white recipes did I try?
The popular Stack Overflow recipe draws each line's outline in white. It got 18.2% of words wrong, about the same as the mask set white.
Lines clear of the text: leave them
When the line sat clear of the letters, Tesseract got 0.1% of words wrong with or without removal. Tables told the same story, on the 73 grid scans from our table extraction guide:
| Method | Best psm | Cells right of 4,575 |
|---|---|---|
| Lines left in | 3 | 3,498 |
| Painted white | 6 | 3,219 |
| Inpaint, the code above | 6 | 3,209 |
| Stack Overflow recipe | 6 | 3,090 |
| OpenCV tutorial sizes | 3 | 2,496 |
| Hough lines | 3 | 1,279 |
Removal does rescue psm 6, which read 87 cells with the lines in. But on these scans, the default psm 3 read the ruled tables better as they were.
How the cells were counted
- Rows and cells rebuilt from the word boxes with the code on the table guide, scored the same way. Lines left in gives that guide's number again.
- Each method ran at psm 3, 4 and 6. The table shows its best.
- On pages painted white, psm 3 read fewer numbers anywhere on the page: 1,928, against 2,165 with the lines in.
Make the length longer than any stroke
length is the shortest run of ink that counts as a line.
The common pick, 40 px, also took the stems of the bold titles above
the tables. Inpaint, psm 6:
| Length | Titles read right of 72 | Cells right |
|---|---|---|
| 40 px | 17 | 3,224 |
| 60 px | 72 | 3,209 |
| 80 px | 72 | 3,076 |
Longer lost cells, for a reason I did not test. I picked 60 on these same pages, so start there and look at the cleaned image before you trust it.
Limits, and what I did not test
- Any straight run of ink at least 60 px long goes, so a solid black bar or a thick rule you wanted goes too.
- Real scans and phone photos. Every page here is made up.
- Dotted, curved or colored lines, and handwriting on lines.
Sources
Our Actor uses the lines first
Our Apify Actor uses the grid lines to mark each cell, then erases them. It reads each cell on its own and checks the row math. On the same 73 scans, an earlier build got 4,570 of the 4,575 cells.
Two ways to skip the work
Run it yourself. Our Apify Actor turns scanned PDFs into Markdown, tables included.
Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free.
Questions people ask
How do I remove horizontal lines from an image in OpenCV?
Threshold the page so ink is white, then open it with a kernel 1 px tall and 60 px wide. Only runs of ink 60 px or longer survive, here the lines. Widen that mask a pixel and give it to cv2.inpaint.
Does removing lines help Tesseract?
Only when the lines touch the letters, and only with inpaint. Lines clear of the text did not hurt Tesseract here, and removing table grid lines lost cells.
Why are my letters broken after I remove the lines?
Painting the mask white also wipes the parts of letters that cross the line. Inpaint fills the mask from the pixels around it, so the strokes come back.