Papermerge OCR, tested on scanned PDFs
0
words read by the README's docker run
The one-line docker run has no OCR
This is the start command in the Papermerge README:
docker run -p 8000:80 \
-e PAPERMERGE__SECURITY__SECRET_KEY=abc \
-e PAPERMERGE__AUTH__PASSWORD=123 \
papermerge/papermerge:3.5
It started, I logged in and sent a 10-page scan. After 5 minutes the document had one version and 0 characters of text.
The image has no Redis and no OCR worker, so the OCR job goes nowhere. With the 3.5.3 tag, the same command fails its SQLite update at start, and every login gets 401.
Fix: run the OCR worker
The docs' compose file adds Postgres, Redis and this worker. I ran it as printed, with 3 small changes, shown below:
ocr_worker:
image: papermerge/ocrworker:0.3.1
platform: linux/amd64
command: worker
environment:
PAPERMERGE__DATABASE__URL: postgresql://coco:jumbo@db:5432/pmgdb
PAPERMERGE__REDIS__URL: redis://redis:6379/0
PAPERMERGE__MAIN__MEDIA_ROOT: /var/media/pmg
OCR_WORKER_ARGS: "-Q ocr -c 2"
depends_on:
- redis
- db
volumes:
- media_root:/var/media/pmg
1,864 of 1,955 true words found
44 to 48 s for 10 pages, on 2 CPUs
A typed cover page in front kept all 13 of its words. The worker runs OCRmyPDF on every page.
The whole compose file that ran
services:
webapp:
image: papermerge/papermerge:3.5.3
platform: linux/amd64
environment:
PAPERMERGE__SECURITY__SECRET_KEY: 12345
PAPERMERGE__AUTH__USERNAME: admin
PAPERMERGE__AUTH__PASSWORD: admin
PAPERMERGE__DATABASE__URL: postgresql://coco:jumbo@db:5432/pmgdb
PAPERMERGE__MAIN__MEDIA_ROOT: /var/media/pmg
PAPERMERGE__REDIS__URL: redis://redis:6379/0
PAPERMERGE__OCR__LANG_CODES: "eng,deu"
PAPERMERGE__OCR__DEFAULT_LANG_CODE: "deu"
volumes:
- media_root:/var/media/pmg
ports:
- "127.0.0.1:12000:80"
depends_on:
- db
- redis
ocr_worker:
image: papermerge/ocrworker:0.3.1
platform: linux/amd64
command: worker
environment:
PAPERMERGE__DATABASE__URL: postgresql://coco:jumbo@db:5432/pmgdb
PAPERMERGE__REDIS__URL: redis://redis:6379/0
PAPERMERGE__MAIN__MEDIA_ROOT: /var/media/pmg
OCR_WORKER_ARGS: "-Q ocr -c 2"
depends_on:
- redis
- db
volumes:
- media_root:/var/media/pmg
db:
image: postgres:16.1
volumes:
- pgdata:/var/lib/postgresql/data/
environment:
POSTGRES_PASSWORD: jumbo
POSTGRES_DB: pmgdb
POSTGRES_USER: coco
healthcheck:
test: pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB
interval: 5s
timeout: 10s
retries: 5
start_period: 10s
redis:
image: redis:7.2
environment:
ALLOW_EMPTY_PASSWORD: "yes"
volumes:
pgdata:
media_root:
My changes: the port on 127.0.0.1, no host port for Redis, and the two platform lines.
On Apple silicon, add a platform line
The Papermerge images are amd64 only. On an M-series Mac:
Error response from daemon: no matching manifest for linux/arm64/v8 in the manifest list entries: no match for platform in manifest: not found
Add platform: linux/amd64 under the webapp and ocr_worker
images. Docker then runs them in emulation. My times above are with
emulation.
Pin the tag, too. A plain
docker pull papermerge/papermerge gets Papermerge 2.1.8,
from April 2023.
The docs' token command fails
The docs say to run create_token.sh in the container. In
3.5.3 it stops with:
could not find task "auth-cli"
A login works. I saved this as token.sh:
curl -s -X POST "$PM/api/token" -H "Content-Type: application/json" \
-d "{\"username\": \"admin\", \"password\": \"$PM_PASSWORD\"}" | jq -r .access_token
Then export PM_TOKEN=$(sh token.sh), with
PM=http://localhost:12000 and the admin password in
PM_PASSWORD.
The default language is German
The docs' file sets DEFAULT_LANG_CODE: "deu". Name the
language on each upload. I saved this as upload.sh:
H="Authorization: Bearer $PM_TOKEN"
INBOX=$(curl -s -H "$H" "$PM/api/users/me" | jq -r .inbox_folder_id)
LANG_JSON=${2:+", \"lang\": \"$2\""}
DOC=$(curl -s -H "$H" -H "Content-Type: application/json" \
-d "{\"title\": \"$1\", \"ctype\": \"document\", \"parent_id\": \"$INBOX\"$LANG_JSON}" \
"$PM/api/nodes/" | jq -r .id)
curl -s -o /dev/null -H "$H" -F "file=@$1" "$PM/api/documents/$DOC/upload"
echo "$DOC"
| Language | 10 English pages | 3 German pages |
|---|---|---|
| deu, the default | 1,852 | 1,845 |
| eng | 1,864 | 1,622 |
With eng, the German pages lost all 206 umlauts (ä, ö, ü, ß). With deu, 192 were right.
Read the text from the OCR PDF
In 3.5.3 the API lists each page with an ID and a number only. The text sits in the database. So read the new PDF version, saved as text.sh:
H="Authorization: Bearer $PM_TOKEN"
URL=$(curl -s -H "$H" "$PM/api/documents/$1/last-version/" | jq -r .download_url)
curl -s -H "$H" -o "$1.pdf" "$PM$URL"
pdftotext -raw "$1.pdf" -
Plain pdftotext found 1,853 words. Its layout mode mixed up the German columns. The text Papermerge stored had 1,879.
Other languages need a new worker
The worker has deu, eng, ron and spa. With fra, OCRmyPDF failed on every page, and the worker log at its default level showed nothing:
OCR engine does not have language data for the following requested languages:
fra
Please install the appropriate language data for your OCR engine.
The docs' fix is your own worker image. This ran:
FROM papermerge/ocrworker:0.3.1
RUN apt update && apt install -y tesseract-ocr-fra
It built in 9 seconds. Then sh ocr.sh with fra gave the
document its OCR version.
Want Markdown out? Use the Actor
Papermerge is free and keeps your files in order. Our OCR Actor gives Markdown with no server. On the same files:
1,874 of 1,955 English words
192 of 206 German umlauts
It also kept the 13 cover words. It costs $0.006 a page.
How I tested it
- Scan: the 10-page PDF of OCR in n8n, proofread Wikisource book scans with no text layer. 1,955 true words.
- Cover page: a 13-word typed sheet in front of the scan, the file of AnythingLLM OCR.
- German: 3 proofread de.wikisource pages, the file of Paperless-ngx OCR. 1,887 words, 206 umlauts.
- Score: true words found in order, with the scorer of the tessdata test.
- Papermerge 3.5.3 and OCR worker 0.3.1 (Tesseract 5.3.0, OCRmyPDF 16.6.2), from the docs' compose file, with Postgres 16.1 and Redis 7.2.
- Mac with Apple silicon, Docker 29.8.2. Worker and webapp on 2 CPUs each. Times are from the upload to the new version.
- The code blocks above are the files that ran. Stored text: read with psql. Actor: real runs on build 0.1.65, Nov 12, from Obsidian OCR.
What I did not test
- Search: the I3 worker with Solr, and the S3 worker.
- The web UI, email import, and the 3.6 alpha images.
- Native arm64 builds, and handwriting.
Sources
Two ways to skip the work
Run it yourself. Our Apify Actor turns scanned PDFs into Markdown, tables included.
Or send us the scan. Tack, our AI, does the job for $29, up to 50 pages. First page free.
Questions people ask
Why does Papermerge not OCR my documents?
The Papermerge app only queues OCR jobs. A separate OCR worker container (papermerge/ocrworker) does the work, and Redis carries the jobs. The one-line docker run has neither, so uploads stay unread.
How do I add an OCR language to Papermerge?
Build a worker image with the Tesseract language pack, for example apt install -y tesseract-ocr-fra on top of papermerge/ocrworker:0.3.1, and run the worker on it. Without the pack, every page fails with MissingDependencyError.
Why does Papermerge search find nothing?
With the docs' OCR worker compose file, GET /api/search/ answered 404 Not Found: that setup has no search engine. The docs' search setups add an I3 worker and Solr. I did not test them.
Related guides
- Docspell OCR, tested on scanned PDFs
- Paperless-ngx OCR, tested on scanned PDFs
- Stirling-PDF OCR, tested on scanned PDFs
- Nextcloud OCR, tested on scanned PDFs
- OCRmyPDF language: set -l, or lose the accents
- Tesseract and German: eng reads no ä, ö, ü or ß
- Tesseract in Docker: 6 base images, tested
- A PDF OCR API in one curl call