Tesseract Japanese OCR: which model reads your scans
82%
characters right with jpn_vert on column scans
jpn reads columns as noise
The first line of a 1991 novel, printed in columns:
| Model | Read |
|---|---|
| Proofread |
『えッ、死んだ。』オットが驚いたように反問した。
|
| jpn_vert, best |
【えッ、死んだ。』オットが驚いたように反問した。
|
| jpn_vert, fast |
[えッ 、 死 ん だ 。』 オ ッ ト が 驚 いた よう に 反 問 し
た 。
|
| jpn | か 国人227』 ボス保護2おでいら直証つる5 |
Both jpn_vert files got every kanji and kana. The fast one put spaces between them.
Columns: jpn_vert reads 4 in 5
Proofread scans from 9 books, 1905 to 2003, in columns. Characters right, of 26,605:
| Model | Size | Right |
|---|---|---|
| jpn | 2.5 MB | 2.2% |
| script/Japanese_vert | 6.4 MB | 75.4% |
| jpn_vert, fast | 3.0 MB | 79.0% |
| jpn_vert, tessdata | 3.0 MB | 81.5% |
| jpn_vert, best | 14.3 MB | 81.6% |
The fast file is the one apt and Homebrew install. best took 2.20 s a page against 1.60 s.
How I tested it
- Pages: ja.wikisource scans whose text was proofread (level 3) or also checked by a second person (level 4). Books printed in 1900 or later, one file per work, picked by a seeded shuffle, not by me.
- Each page is a real scan: a DjVu file, or a PDF page that holds one full-page image. Pages were 1920 px wide. I checked the line direction of every page by eye.
-
One plain
tesseractrun per page and model, default settings, Tesseract 5.5.4. - A character counts when it is found in order. Only kanji and kana count, not punctuation, digits or Latin letters.
What about furigana?
- 18 of the 35 pages have furigana. I removed it from the proofread text, so a model is not asked to read it.
- Furigana read as text costs nothing here, because extra characters are not counted.
- The 1918 book with furigana beside almost every kanji was hard: best got 65.8%, and the fast file lost the layout (2.8%).
--psm 5 made it worse
--psm 5 reads the page as one block of vertical text:
79.0% fast jpn_vert, default
68.6% with --psm 5
It raised 6 of the 9 books, but pages printed in tiers lost far more:
| Book | Default | psm 5 |
|---|---|---|
| 2003 diary, two tiers | 79.6% | 52.8% |
| 1926 magazine, tiers | 77.1% | 32.8% |
| 1918 book, furigana | 2.8% | 78.2% |
Try it on one page before you run a whole book with it.
Lines across: jpn, not jpn_vert
Only 3 books of horizontal scans passed the checks, 1911 to 2018, so treat this as a rough check. Characters right, of 5,590:
| Model | Size | Right |
|---|---|---|
| jpn_vert | 3.0 MB | 1.7% |
| jpn, best | 14.3 MB | 52.5% |
| jpn, fast | 2.5 MB | 59.4% |
| script/Japanese | 5.9 MB | 61.6% |
Spaces between characters
The fast jpn_vert puts a space between characters. One flag fixes it:
19,708 spaces with the fast jpn_vert
206 with
-c preserve_interword_spaces=1
Not one character changed. The best jpn_vert adds almost none.
Install the models and run them
apt (tesseract-ocr-jpn, tesseract-ocr-jpn-vert) and Homebrew (tesseract-lang) install the fast files. To pick your own, put them in a folder:
mkdir -p ~/tessdata
curl -L -o ~/tessdata/jpn_vert.traineddata \
https://raw.githubusercontent.com/tesseract-ocr/tessdata_best/4.1.0/jpn_vert.traineddata
curl -L -o ~/tessdata/jpn.traineddata \
https://raw.githubusercontent.com/tesseract-ocr/tessdata_fast/4.1.0/jpn.traineddata
tesseract page.png page --tessdata-dir ~/tessdata -l jpn_vert
In Python, with pytesseract. For lines across, use jpn:
import os
import pytesseract
TESSDATA = os.path.expanduser("~/tessdata")
text = pytesseract.image_to_string(
"page.png",
lang="jpn_vert",
config=f"--tessdata-dir {TESSDATA} -c preserve_interword_spaces=1",
)
What I did not test
- Windows and Linux. One Mac, Tesseract 5.5.4.
- Woodblock prints, manuscripts and handwriting.
- Screenshots and photos of signs or menus.
- Many horizontal books: only 3 here.
- Cleaning the image first, and other OCR engines.
Sources
Our Actor reads lines, not columns
Our Apify Actor has the same fast jpn. On the horizontal scans it got 67.2% of the characters right. It has no jpn_vert: on the column scans it got 2.3%.
Two ways to skip the work
Run it yourself. Our Apify Actor turns scanned PDFs into Markdown, tables included.
Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free.
Questions people ask
How do I OCR vertical Japanese with Tesseract?
Use jpn_vert, not jpn. On 35 proofread column scans, jpn_vert best got 81.6% of the characters right and jpn got 2.2%.
Does --psm 5 help with vertical Japanese?
Not on these scans. It took the fast jpn_vert from 79.0% to 68.6%. It hurt most on pages printed in tiers.
How do I add Japanese to Tesseract?
Install the jpn files: apt install tesseract-ocr-jpn tesseract-ocr-jpn-vert, or brew install tesseract-lang. Then pass -l jpn for lines or -l jpn_vert for columns.