Tools,

Manga OCR in Python, tested on 160 bubbles

23 of 40

vertical bubbles exact, only with manga-ocr

Get the code

manga-ocr reads one bubble a call

pip install manga-ocr
A white speech bubble on dot tone with two columns of vertical Japanese text: おのれ、どこへ行く。 in brackets.
bubble.png, one of the test bubbles.
from manga_ocr import MangaOcr

mocr = MangaOcr()
print(mocr("bubble.png"))
「おのれ、どこへ行く。」

The first run downloads about 890 MB of model files. On a Mac it uses the GPU by itself: 0.27 s a bubble, or 0.75 s with force_cpu=True.

Columns: manga-ocr 23 exact, the rest 0

40 vertical bubbles
Way Characters right Exact
manga-ocr 0.1.16 96.8% 23
RapidOCR, columns sorted 98.1% 0
RapidOCR, as it comes 33.5% 0
Tesseract, jpn_vert best 89.1% 0
Tesseract, jpn_vert fast 72.4% 0
Tesseract, jpn 6.3% 0

Characters right counts kanji and kana in order. Exact means the whole bubble, punctuation included. At half size as a JPEG, manga-ocr still read 20 of the 40 columns exactly.

9 of manga-ocr's 17 misses were one character, such as a dakuten (ぎ read as き). Others were look-alikes (土 as 士) or rare kanji, such as 蟋蟀 (cricket) read as 蝶蜂.

RapidOCR reads kanji, loses commas

RapidOCR reads each column, but left to right. Sorted right to left, it got the most kanji and kana: 24 bubbles matched on them. It lost 73 of 79 commas, so none was exact.

from rapidocr import RapidOCR

result = RapidOCR()("bubble.png")
# Japanese columns run right to left: sort by each box's right edge
columns = sorted(zip(result.boxes, result.txts), key=lambda c: -c[0][:, 0].max())
print("".join(text for _, text in columns))
。pOOOp++GOGGOOOOOOOOOOOOOOOO自「おのれ1行くoOOOOOO++

Letters and digits came out in 5 of the 40 bubbles: dot tone as O and +, and some 、 and 。 as 1 and o, as here.

Lines across: RapidOCR wins

The same 40 lines, written across
Way Characters right Exact
RapidOCR 97.4% 24
Tesseract, jpn 94.9% 19
manga-ocr 95.7% 18

A whole page: words that are not there

from manga_ocr import MangaOcr

mocr = MangaOcr()
print(mocr("page.png"))
それでも、その結果、

Only それでも is on page 1. On page 2 it wrote 購入できるのは、 (what you can buy is), which is on neither page. manga-ocr expects one text box and always returns one line of text.

Cut the bubbles out first

pip install manga-ocr rapidocr onnxruntime
import cv2
import numpy as np
from manga_ocr import MangaOcr
from PIL import Image
from rapidocr import RapidOCR

page = Image.open("page.png")
boxes = RapidOCR()("page.png", use_det=True, use_cls=False, use_rec=False).boxes
mask = np.zeros((page.height, page.width), np.uint8)
for (x0, y0), (x1, y1) in zip(boxes.min(1) - 12, boxes.max(1) + 12):
    cv2.rectangle(mask, (int(x0), int(y0)), (int(x1), int(y1)), 255, -1)

mocr = MangaOcr()
for x, y, w, h, _ in cv2.connectedComponentsWithStats(mask)[2][1:]:
    print(mocr(page.crop((x, y, x + w, y + h))))

Each text column grows by 12 px, so the columns of one bubble touch and join. It found all 12 bubbles and read 11 exactly, top to bottom, not in manga order.

Furigana: manga-ocr leaves it out

0 furigana characters in manga-ocr's text

89 in RapidOCR's, columns sorted

20 more bubbles, drawn twice: plain, and with the readings beside the kanji. Tesseract's best model added 49 characters.

ValueError with an OpenCV image

import numpy as np
from manga_ocr import MangaOcr
from PIL import Image

mocr = MangaOcr()
frame = np.array(Image.open("bubble.png"))  # what OpenCV and screen grabs give you
print(mocr(frame))
ValueError: img_or_path must be a path or PIL.Image, instead got: [[[ 90  90  90]

manga-ocr takes a path or a PIL image, not an array. Wrap it: mocr(Image.fromarray(frame)).

The manga_ocr command watches

mkdir -p shots
manga_ocr --read_from=shots --write_to=text.txt

It reads only images that arrive after it starts: 2 already in shots were skipped, 3 new ones were read. With no arguments it reads the clipboard and writes back to it.

A write_to name without .txt stops it at the first image:

ValueError: write_to must be either "clipboard" or a path to a text file

Need text from scanned PDFs?

Our Actor reads scanned PDF pages with Tesseract. Its Japanese model is jpn, not jpn_vert, so it is the wrong tool for manga.

On the 2 test pages it got 47 of 293 characters.

curl -s -X POST "https://api.apify.com/v2/acts/brasspage~pdf-ocr-to-markdown/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"documentUrls": ["https://api.apify.com/v2/key-value-stores/sTnmNrGoSsi7tLQvs/records/anythingllm-mixed-11-pages.pdf"]}' \
  | jq -r '.[].markdown' > book.md

It costs $0.006 a page.

How I tested it
  • 40 sentences from Akutagawa's Rashomon (1915, public domain, from Aozora Bunko), picked by a seeded shuffle. Drawn as bubbles on dot tone in Noto Sans JP, Noto Serif JP, Klee One and Mochiy Pop One (all OFL).
  • Columns of up to 8 characters at 28 px, the same lines written across, and a half-size JPEG copy. 20 more sentences with the book's furigana.
  • No real manga pages: I found none whose license lets me publish crops. Drawn text like this is part of manga-ocr's own training, so the test is kind to it.
  • manga-ocr 0.1.16, RapidOCR 3.10.0 and Tesseract 5.5.4 (tessdata_fast and tessdata_best 4.1.0, --psm 5 for jpn_vert) on Python 3.12, Apple Silicon Mac. 2 threads, lowest priority.
  • Akutagawa's long dash is not scored: it looks the same as ーー in a column. The Actor result comes from its own code, run on this Mac with the same jpn model as its image.
What I did not test
  • Real manga scans, color pages, sound effects, text outside bubbles.
  • Apple Live Text, Google Lens, and other OCR models.
Sources

Two ways to skip the work

Run it yourself. Our Apify Actor turns scanned PDFs into Markdown, tables included.

Try the Actor

Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free.

Get one page free

Questions people ask

Can manga-ocr read a whole manga page?

No, it reads one text box and returns one line. On the test pages it returned mostly words that are on neither page. Cut the bubbles out first: that read 11 of 12 bubbles exactly.

Does manga-ocr read furigana?

No, it leaves the small readings out. Furigana added 0 characters to its text. RapidOCR added 89 characters of readings on 20 bubbles.

Why does manga-ocr say img_or_path must be a path or PIL.Image?

It takes a file path or a PIL image, not a NumPy array, which is what OpenCV gives you. Wrap the array in Image.fromarray first.

Does manga-ocr need a GPU?

No. On a Mac it uses the GPU by itself, at 0.27 s a bubble. With force_cpu=True it took 0.75 s a bubble on 2 CPU threads.