Malayalam OCR: six free Tesseract runs, tested
90.8%
of letters, with Tesseract's script/Malayalam
Five models, under a point apart
We ran Tesseract 5.5.4 on 29 proofread pages from 10 printed books on ml.wikisource.
| Model | Letters | Words | Words, fixed |
|---|---|---|---|
| script/Malayalam | 90.8% | 44.9% | 58.1% |
| script/Malayalam best | 90.4% | 44.3% | 55.2% |
| mal, tessdata repo | 90.0% | 43.9% | 55.1% |
| mal best | 89.9% | 43.6% | 55.1% |
| mal | 89.9% | 42.1% | 55.3% |
| mal+eng | 87.7% | 41.7% | 54.8% |
The five Malayalam models sit within 0.9 points of each other.
The fast mal took a median 2.0 s a page. script/Malayalam took 2.3 s.
Letters are proofread letters found in order. Words are whole words right. Fixed is after the chillu fix below.
The command: one model file
curl -LO https://raw.githubusercontent.com/tesseract-ocr/tessdata_fast/4.1.0/script/Malayalam.traineddata
mkdir -p tessdata && mv Malayalam.traineddata tessdata/
tesseract page.png page --tessdata-dir tessdata -l Malayalam
It writes page.txt. The model file is 9 MB.
Homebrew's brew install tesseract-lang has the fast
mal. Use -l mal (89.9% here).
Tesseract types chillus the old way
A chillu, like ൻ, is one letter in Unicode. Tesseract mostly writes it the old way: consonant, virama and an invisible joiner, U+200D.
| Written as | In the books | script/Malayalam |
|---|---|---|
| One letter, like ൻ | 741 | 46 |
| Consonant + virama + U+200D | 0 | 663 |
They look the same. A plain search, like grep, does not match them.
A second old habit: ൻറ for ന്റ. The books hold 79 ന്റ. script/Malayalam wrote ന്റ 13 times and ൻറ or ൻെറ 60 times.
python3 -c 'import re,sys
t=sys.stdin.read()
t=re.sub("([ണനരലളക])\u0d4d\u200d",lambda m:"ൺൻർൽൾൿ"["ണനരലളക".index(m[1])],t)
sys.stdout.write(re.sub("ൻ([െേൈ]?)റ",r"ന്റ\1",t))' < page.txt > fixed.txt
This rewrites both. Every run gains 11 to 13 points of whole words.
The book matters more than the model
Across all 29 pages, the five Malayalam models sat under a point apart. Across books, script/Malayalam moved 10 points.
| Printed | Letters | Words |
|---|---|---|
| 1892 | 85.5% | 27.3% |
| 1903 | 86.9% | 30.7% |
| 1903 | 88.0% | 35.2% |
| 1916 | 90.4% | 40.4% |
| 1928 | 89.8% | 40.1% |
| 1938 | 95.3% | 60.6% |
| 1947 | 90.3% | 43.3% |
| No year given | 90.9% | 54.6% |
| No year given | 88.5% | 54.6% |
| No year given | 95.7% | 57.4% |
The 1892 and 1903 books read worst.
Why so few whole words? Our Malayalam words average 10.6 letters and signs, and one wrong letter spoils a word.
Four letters no model can write
The character sets of all five Malayalam models lack these. 66 of our 2,770 proofread words hold one, so no run gets them right.
| Letter | Times | Books |
|---|---|---|
| ൎ U+0D4E, dot reph | 46 | 5 |
| ഊ U+0D0A | 13 | 2 |
| ൗ U+0D57, on its own | 7 | 6 |
| ഢ U+0D22 | 1 | 1 |
The dot reph is how older books print ർ before a consonant.
A scanned PDF: search the sidecar
mkdir -p tessdata
cp -r "$(brew --prefix)/share/tessdata/configs" tessdata/
curl -L -o tessdata/Malayalam.traineddata \
https://raw.githubusercontent.com/tesseract-ocr/tessdata_fast/4.1.0/script/Malayalam.traineddata
TESSDATA_PREFIX=$PWD/tessdata ocrmypdf -l Malayalam --jobs 2 \
--sidecar book.txt scan.pdf book.pdf
On a 29-page PDF of the same scans it took 64 s. book.txt held 90.4% of letters.
| From | Letters in order | Words |
|---|---|---|
| book.txt | 90.4% | 45.0% |
| book.pdf, pdftotext | 78.2% | 30.9% |
| book.pdf, pdftotext -raw | 90.4% | 31.7% |
| book.pdf, PDFKit (Preview) | 85.2% | 34.1% |
pdftotext -raw puts spaces inside words: 3,484 spaces, where book.txt has 2,295.
Preview drops letters around vowel signs: 277 െ, where book.txt has 666.
Search book.txt after the fix (book.txt in place of page.txt). Without
the cp line (Homebrew), OCRmyPDF stops with
TesseractConfigError.
Three more things we checked
- mal+eng wrote 648 Latin letters on 26 pages that have none. Leave off +eng: mal, mal best and mal std wrote none.
- Handwriting: on 6 pages from two manuscripts at Tübingen, no run read more than 30.3% of letters or 2.8% of words.
- Apple Vision on macOS 26.6 lists 30 languages, not Malayalam.
How we ran it
- Pages marked proofread on ml.wikisource, from a seeded shuffle of books, 3 a book (2 for one book). Pages with footnotes, tables or running heads in the text were left out.
- The 15 handwritten books in that shuffle (Tübingen shelfmark Ci XIV) were left out. 6 pages from 2 of them went to the handwriting test.
- 14 pages came from PDFs and 15 from DjVu files, drawn at most 1,920 px wide.
- Scores count Malayalam letters and signs. Digits, punctuation, Latin letters and spaces are left out.
- Letters found in order is the longest common subsequence. Fixed words apply the fix to the OCR text only.
- Models from tessdata_fast, tessdata_best and tessdata at tag 4.1.0.
- One thread a run, 2 runs at a time, at low priority, while other jobs ran. Times are rough.
- OCRmyPDF 17.13.0, pdftotext from Poppler 26.03.0, PDFKit on macOS 26.6. The PDF holds the 29 scans as JPEGs, one a page.
- Licenses: Tesseract and tessdata are Apache-2.0, OCRmyPDF is MPL-2.0. 4 scan files are on Commons (3 public domain, 1 CC BY-SA 2.5), 5 on ml.wikisource marked PD-India, 1 with no tag.
- The 2 manuscripts are public domain on Commons. We show no page images.
Scanned pages to text, no install
Run it yourself. Our Apify Actor turns a scanned PDF into Markdown or text. $0.006 a scanned page. It reads 18 languages, not Malayalam yet.
Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free. For Malayalam, ask first.
Questions people ask
Does Tesseract support Malayalam?
Yes, with mal or script/Malayalam. On our 29 pages, script/Malayalam read 90.8% of letters, the fast mal 89.9%.
Why does search miss words in my Malayalam OCR text?
Tesseract writes most chillus as a consonant, a virama and U+200D, not as one letter like ൻ. script/Malayalam wrote 663 that way and 46 as one letter. Convert them with the fix on this page.
Is there a free Malayalam OCR?
Tesseract and its models are free (Apache-2.0) and run offline. OCRmyPDF (MPL-2.0) adds a text layer to a scanned PDF.
Can I OCR a Malayalam PDF?
Yes. OCRmyPDF with -l Malayalam found 90.4% of letters on our 29-page PDF. Search its --sidecar text: it had 45.0% of words right, the PDF text layer 34.1% at most.
Can Tesseract read Malayalam handwriting?
Not well. On 6 handwritten manuscript pages, no run read more than 30.3% of letters or 2.8% of words.