Tools,

Malayalam OCR: six free Tesseract runs, tested

90.8%

of letters, with Tesseract's script/Malayalam

Get the command

Five models, under a point apart

We ran Tesseract 5.5.4 on 29 proofread pages from 10 printed books on ml.wikisource.

29 pages: letters and whole words right
Model Letters Words Words, fixed
script/Malayalam 90.8% 44.9% 58.1%
script/Malayalam best 90.4% 44.3% 55.2%
mal, tessdata repo 90.0% 43.9% 55.1%
mal best 89.9% 43.6% 55.1%
mal 89.9% 42.1% 55.3%
mal+eng 87.7% 41.7% 54.8%

The five Malayalam models sit within 0.9 points of each other.

The fast mal took a median 2.0 s a page. script/Malayalam took 2.3 s.

Letters are proofread letters found in order. Words are whole words right. Fixed is after the chillu fix below.

The command: one model file

curl -LO https://raw.githubusercontent.com/tesseract-ocr/tessdata_fast/4.1.0/script/Malayalam.traineddata
mkdir -p tessdata && mv Malayalam.traineddata tessdata/
tesseract page.png page --tessdata-dir tessdata -l Malayalam

It writes page.txt. The model file is 9 MB.

Homebrew's brew install tesseract-lang has the fast mal. Use -l mal (89.9% here).

Tesseract types chillus the old way

A chillu, like ൻ, is one letter in Unicode. Tesseract mostly writes it the old way: consonant, virama and an invisible joiner, U+200D.

29 pages: chillus in the books, and in script/Malayalam
Written as In the books script/Malayalam
One letter, like ൻ 741 46
Consonant + virama + U+200D 0 663

They look the same. A plain search, like grep, does not match them.

A second old habit: ൻറ for ന്റ. The books hold 79 ന്റ. script/Malayalam wrote ന്റ 13 times and ൻറ or ൻെറ 60 times.

python3 -c 'import re,sys
t=sys.stdin.read()
t=re.sub("([ണനരലളക])\u0d4d\u200d",lambda m:"ൺൻർൽൾൿ"["ണനരലളക".index(m[1])],t)
sys.stdout.write(re.sub("ൻ([െേൈ]?)റ",r"ന്റ\1",t))' < page.txt > fixed.txt

This rewrites both. Every run gains 11 to 13 points of whole words.

The book matters more than the model

Across all 29 pages, the five Malayalam models sat under a point apart. Across books, script/Malayalam moved 10 points.

Each book, script/Malayalam
Printed Letters Words
1892 85.5% 27.3%
1903 86.9% 30.7%
1903 88.0% 35.2%
1916 90.4% 40.4%
1928 89.8% 40.1%
1938 95.3% 60.6%
1947 90.3% 43.3%
No year given 90.9% 54.6%
No year given 88.5% 54.6%
No year given 95.7% 57.4%

The 1892 and 1903 books read worst.

Why so few whole words? Our Malayalam words average 10.6 letters and signs, and one wrong letter spoils a word.

Four letters no model can write

The character sets of all five Malayalam models lack these. 66 of our 2,770 proofread words hold one, so no run gets them right.

29 pages: in the books
Letter Times Books
ൎ U+0D4E, dot reph 46 5
ഊ U+0D0A 13 2
ൗ U+0D57, on its own 7 6
ഢ U+0D22 1 1

The dot reph is how older books print ർ before a consonant.

A scanned PDF: search the sidecar

mkdir -p tessdata
cp -r "$(brew --prefix)/share/tessdata/configs" tessdata/
curl -L -o tessdata/Malayalam.traineddata \
  https://raw.githubusercontent.com/tesseract-ocr/tessdata_fast/4.1.0/script/Malayalam.traineddata
TESSDATA_PREFIX=$PWD/tessdata ocrmypdf -l Malayalam --jobs 2 \
  --sidecar book.txt scan.pdf book.pdf

On a 29-page PDF of the same scans it took 64 s. book.txt held 90.4% of letters.

The same text, read back four ways
From Letters in order Words
book.txt 90.4% 45.0%
book.pdf, pdftotext 78.2% 30.9%
book.pdf, pdftotext -raw 90.4% 31.7%
book.pdf, PDFKit (Preview) 85.2% 34.1%

pdftotext -raw puts spaces inside words: 3,484 spaces, where book.txt has 2,295.

Preview drops letters around vowel signs: 277 െ, where book.txt has 666.

Search book.txt after the fix (book.txt in place of page.txt). Without the cp line (Homebrew), OCRmyPDF stops with TesseractConfigError.

Three more things we checked

How we ran it
  • Pages marked proofread on ml.wikisource, from a seeded shuffle of books, 3 a book (2 for one book). Pages with footnotes, tables or running heads in the text were left out.
  • The 15 handwritten books in that shuffle (Tübingen shelfmark Ci XIV) were left out. 6 pages from 2 of them went to the handwriting test.
  • 14 pages came from PDFs and 15 from DjVu files, drawn at most 1,920 px wide.
  • Scores count Malayalam letters and signs. Digits, punctuation, Latin letters and spaces are left out.
  • Letters found in order is the longest common subsequence. Fixed words apply the fix to the OCR text only.
  • Models from tessdata_fast, tessdata_best and tessdata at tag 4.1.0.
  • One thread a run, 2 runs at a time, at low priority, while other jobs ran. Times are rough.
  • OCRmyPDF 17.13.0, pdftotext from Poppler 26.03.0, PDFKit on macOS 26.6. The PDF holds the 29 scans as JPEGs, one a page.
  • Licenses: Tesseract and tessdata are Apache-2.0, OCRmyPDF is MPL-2.0. 4 scan files are on Commons (3 public domain, 1 CC BY-SA 2.5), 5 on ml.wikisource marked PD-India, 1 with no tag.
  • The 2 manuscripts are public domain on Commons. We show no page images.

Scanned pages to text, no install

Run it yourself. Our Apify Actor turns a scanned PDF into Markdown or text. $0.006 a scanned page. It reads 18 languages, not Malayalam yet.

Try the Actor

Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free. For Malayalam, ask first.

Get one page free

Questions people ask

Does Tesseract support Malayalam?

Yes, with mal or script/Malayalam. On our 29 pages, script/Malayalam read 90.8% of letters, the fast mal 89.9%.

Why does search miss words in my Malayalam OCR text?

Tesseract writes most chillus as a consonant, a virama and U+200D, not as one letter like ൻ. script/Malayalam wrote 663 that way and 46 as one letter. Convert them with the fix on this page.

Is there a free Malayalam OCR?

Tesseract and its models are free (Apache-2.0) and run offline. OCRmyPDF (MPL-2.0) adds a text layer to a scanned PDF.

Can I OCR a Malayalam PDF?

Yes. OCRmyPDF with -l Malayalam found 90.4% of letters on our 29-page PDF. Search its --sidecar text: it had 45.0% of words right, the PDF text layer 34.1% at most.

Can Tesseract read Malayalam handwriting?

Not well. On 6 handwritten manuscript pages, no run read more than 30.3% of letters or 2.8% of words.