Telugu PDF to Unicode

How to get correct Telugu out of a PDF

1. Choose the PDF (a locked one asks for its password, which stays on your device).
2. Each page is checked at once, and you see how its text will be taken: Text in the PDF, Old font converted or Read with OCR, and why. You can choose OCR for any page, or keep a page’s own text.
3. Choose the pages, and plain text or Markdown.
4. Tap Get the Telugu as Unicode, correct anything in the text box, then Copy, Share, or save it as .txt, .md or Word.

కాపీ చేస్తే తెలుగు అక్షరాలు తప్పుగా వచ్చే PDF (జీవోలు, మీసేవ ఫారాలు) ఎంచుకోండి: సరైన తెలుగు యూనికోడ్ వస్తుంది.

Why the Telugu comes out garbled

• Old 8-bit fonts such as Anu, Priyaanka or Kranthi draw Telugu in the places of English letters, so a copy gives letters like Mì`Œ^¥~¡°.
• Unicode fonts saved badly: many PDFs made from Word with Gautami or Nirmala UI give their letters back out of order, with spaces inside words or with the vottu letters missing ( వ్ యక్తి, సంఖయ for సంఖ్య). This is the most common case in government and bank forms.
• No text at all: a scan, or a form whose Telugu was turned into shapes when it was made.

How each page is taken

• Text in the PDF: its Telugu is proper Unicode (it is checked word by word against the 60,000 commonest Telugu words and Unicode’s spelling rules), or the page is English. Small slips, such as a space inside a word, are mended.
• Old font converted: Telugu in an Anu-family font, found by the font’s name or by its letters, made Unicode with SIL International’s Anu map and ap7am’s corrections (the map behind our Anu to Unicode converter), its vowel signs and conjuncts put in Unicode’s order. Each letter’s code is taken from how the page draws it, not from the PDF’s copy text, which often leaves letters out; letters a PDF leaves unnamed are worked out from the words they are in.
• From the PDF’s letters: garbled Unicode (most often Word’s PDFs in Gautami) whose font, embedded in the PDF, says what each of its glyphs is: every letter is read from the font itself and put back in order, so the text is exact, with no OCR. Only when nearly every letter on the page is known this way; otherwise the page is read with OCR and says why.
• Read with OCR: a scan, Telugu drawn as shapes, garbled Unicode, or an old font with no map here (or one whose letters cannot all be worked out). The page is drawn at 300 dots an inch, cleaned (the lines of a form’s boxes removed) and read by Tesseract in Telugu and English, the right way up.

On OCR pages, common misreads are fixed with Telugu Typing’s word list, and Proofread shows each line under the strip of the page it was read from, to check and fix (your fixes are remembered on this device only; “Forget my fixes” clears them).

English words stay English. The text comes out in reading order, with its paragraphs and headings; simple tables as Markdown tables, or with their cells apart by tabs in plain text (they paste into Excel); in Word, as tables, with Telugu set in Nirmala UI.

Better Telugu reading (a switch, on by default on computers) has a second Telugu reader look again at the lines Tesseract was unsure of, and keeps the better word of the two, using Telugu Typing’s word list. On our hand-checked pages it took the Telugu words read wrong or missed from 57 in 749 to 25; it makes reading up to three times as long and needs up to about 250 MB more memory, so it starts off on phones. It downloads about 22 MB once, from ap7am, and is kept with the models; if it cannot run (an older browser, too little memory), the text is read with Tesseract alone and the page says so.

Everything happens in your browser: the PDF is never uploaded. When a page needs OCR, the reader and its models (about 6 MB) download from ap7am once and are kept in this browser, shared with OCR PDF and Image to Text.

What it cannot do

• OCR makes mistakes, more often in Telugu: check names, numbers and dates against the PDF. Pages taken from the PDF’s own text or converted from Anu have none of OCR’s mistakes, but check rare letters.
• Only Anu-family fonts have a map. Other old fonts (Shree, TLW and the like, and Anu fonts made on a Mac, which put letters in other places) are read with OCR.
• Handwriting, stamps and signatures are not read; complex tables may come out as lines.
• The result is text, not a copy of the PDF’s look. To keep the scan and make it searchable, use OCR PDF.

Questions

Made with SIL International’s Anu (Telegu Anu) TECkit map (MIT licence), Tesseract OCR (tesseract.js and tesseract.js-core, Apache 2.0 licence) and its Telugu and English models (Apache 2.0, packaged under the MIT licence), harsha-desaraju’s Telugu OCR model (MIT licence) and ONNX Runtime Web (MIT licence) for Better Telugu reading, and Mozilla’s pdf.js (Apache 2.0 licence), all running in your browser. Licences.