Skill Tracer
Skills › Scientific Agent Skills › liteparse › ocr_and_formats.md

ocr_and_formats.md

135 lines · skills/liteparse/references/ocr_and_formats.md at 5f2bc03

Open on GitHub
1# OCR and Supported Input Formats (2.15.0)
2

OCR and Supported Input Formats (2.15.0)

Raw lines 1 to 2
3## Built-in Tesseract
4
5OCR is enabled by default and runs selectively on sparse pages and embedded
6images. `--no-ocr` / `ocr_enabled=False` bypasses it. Tesseract itself is bundled,
7but a missing language file triggers a download, even if `tessdata_path` or
8`TESSDATA_PREFIX` names a directory. The client performs an unauthenticated GET
9of `https://github.com/tesseract-ocr/tessdata_best/raw/main/<code>.traineddata`.
11For offline operation, populate **every** requested file before parsing and use
12an explicit directory; merely setting that directory does not disable downloads.
13`ocr_language="eng+fra"` requires both files. Standard Tesseract codes include
14`eng`, `fra`, `deu`, `spa`, `chi_sim`; common two-letter aliases are normalized
15by the built-in engine. Language data and download permissions are separate
16from installing the package.
18```bash
19lit parse scan.pdf --ocr-language eng --tessdata-path ./tessdata
20lit parse document.pdf --no-ocr
21```
23Check that `./tessdata/eng.traineddata` exists before the first command. These
24commands assume caller-supplied input files; local English OCR was exercised
25with already-installed language data, not a network download.

Built-in Tesseract

OCR is enabled by default and runs selectively on sparse pages and embedded images. --no-ocr / ocr_enabled=False bypasses it. Tesseract itself is bundled, but a missing language file triggers a download, even if tessdata_path or TESSDATA_PREFIX names a directory. The client performs an unauthenticated GET of https://github.com/tesseract-ocr/tessdata_best/raw/main/<code>.traineddata.

For offline operation, populate every requested file before parsing and use an explicit directory; merely setting that directory does not disable downloads. ocr_language="eng+fra" requires both files. Standard Tesseract codes include eng, fra, deu, spa, chi_sim; common two-letter aliases are normalized by the built-in engine. Language data and download permissions are separate from installing the package.

lit parse scan.pdf --ocr-language eng --tessdata-path ./tessdata
lit parse document.pdf --no-ocr

Check that ./tessdata/eng.traineddata exists before the first command. These commands assume caller-supplied input files; local English OCR was exercised with already-installed language data, not a network download.

Raw lines 3 to 26
27## Optional HTTP OCR
29Pass the **complete endpoint URL**, including its path. LiteParse POSTs to the
30URL unchanged; it does **not** append `/ocr`:
32```bash
33lit parse scan.pdf --ocr-server-url http://localhost:8080/ocr --ocr-language en
34```
36This sends rendered document images to that server. There is no built-in cloud
37account, fixed hosted endpoint, or universal auth scheme. A server requiring
38credentials can be configured through Python without embedding a secret:
40```python
41import os
42from liteparse import LiteParse
44parser = LiteParse(
45 ocr_server_url=os.environ["OCR_SERVER_URL"],
46 ocr_server_headers={"Authorization": "Bearer " + os.environ["OCR_API_KEY"]},
47 ocr_language="en",
48 quiet=True,
50result = parser.parse("scan.pdf")
51```
53This authentication example is illustrative; configure the header scheme your
54server requires. CLI `--ocr-server-header "Name: Value"` is repeatable. Do not
55log credentials; command arguments may be visible in process lists.

Optional HTTP OCR

Pass the complete endpoint URL, including its path. LiteParse POSTs to the URL unchanged; it does not append /ocr:

lit parse scan.pdf --ocr-server-url http://localhost:8080/ocr --ocr-language en

This sends rendered document images to that server. There is no built-in cloud account, fixed hosted endpoint, or universal auth scheme. A server requiring credentials can be configured through Python without embedding a secret:

import os
from liteparse import LiteParse

parser = LiteParse(
    ocr_server_url=os.environ["OCR_SERVER_URL"],
    ocr_server_headers={"Authorization": "Bearer " + os.environ["OCR_API_KEY"]},
    ocr_language="en",
    quiet=True,
)
result = parser.parse("scan.pdf")

This authentication example is illustrative; configure the header scheme your server requires. CLI --ocr-server-header "Name: Value" is repeatable. Do not log credentials; command arguments may be visible in process lists.

Raw lines 27 to 56
57### Wire contract
59- Method: **POST** to the exact configured URL (commonly `/ocr`). No pagination,
60 jobs, polling, or separate upload endpoint: each call processes one raster.
61- Request: `multipart/form-data`; `file` is PNG bytes named `image.png`, and
62 `language` is a string.
63- **Language is forwarded unchanged to HTTP OCR.** The standalone API spec
64 suggests ISO 639-1 (`en`, `fr`), while LiteParse's default is `eng`. Set a code
65 accepted by the server; the HTTP client does not perform the Tesseract alias mapping.
66- Response: HTTP 200, `application/json`, with `results` entries containing
67 `text`, `bbox` and `confidence`. Empty results are valid.
69```json
71 "results": [{
72 "text": "recognized text",
73 "bbox": [10, 20, 130, 40],
74 "confidence": 0.95
75 }]
77```
79Coordinates are top-left-origin image pixels; `[x1,y1,x2,y2]` must have positive
80width/height. Optional `polygon` is four `[x,y]` points ordered TL, TR, BR, BL in
81the glyphs' upright reading frame, to preserve rotation information.
83The client also accepts a worker-style `result` array of `[polygon,text,confidence]`
84tuples, but use the standard `results` contract for new servers. Non-2xx HTTP
85responses fail. In 2.15.0, transient failures can trigger up to 10 attempts with
8660-second per-request timeouts and backoff; 400/401/404 are not retried.
87`ocr_failure_fatal=True` (default) fails the parse if all OCR tasks fail and at
88least one sparse page depends on OCR. Partial OCR failures may still leave an
89incomplete document: compare source pages and extracted content. Python
90`pool_size` plus `parse_timeout` can enforce a whole-parse deadline. Request
91hedging is opt-in and creates duplicate server work; leave it unset normally.
93A loopback test verified exact URL, multipart PNG, language, auth header,
94successful result decoding and non-retried 401 handling. This verifies client
95transport, not any production OCR service or its recognition quality.

Wire contract

  • Method: POST to the exact configured URL (commonly /ocr). No pagination, jobs, polling, or separate upload endpoint: each call processes one raster.
  • Request: multipart/form-data; file is PNG bytes named image.png, and language is a string.
  • Language is forwarded unchanged to HTTP OCR. The standalone API spec suggests ISO 639-1 (en, fr), while LiteParse's default is eng. Set a code accepted by the server; the HTTP client does not perform the Tesseract alias mapping.
  • Response: HTTP 200, application/json, with results entries containing text, bbox and confidence. Empty results are valid.
{
  "results": [{
    "text": "recognized text",
    "bbox": [10, 20, 130, 40],
    "confidence": 0.95
  }]
}

Coordinates are top-left-origin image pixels; [x1,y1,x2,y2] must have positive width/height. Optional polygon is four [x,y] points ordered TL, TR, BR, BL in the glyphs' upright reading frame, to preserve rotation information.

The client also accepts a worker-style result array of [polygon,text,confidence] tuples, but use the standard results contract for new servers. Non-2xx HTTP responses fail. In 2.15.0, transient failures can trigger up to 10 attempts with 60-second per-request timeouts and backoff; 400/401/404 are not retried. ocr_failure_fatal=True (default) fails the parse if all OCR tasks fail and at least one sparse page depends on OCR. Partial OCR failures may still leave an incomplete document: compare source pages and extracted content. Python pool_size plus parse_timeout can enforce a whole-parse deadline. Request hedging is opt-in and creates duplicate server work; leave it unset normally.

A loopback test verified exact URL, multipart PNG, language, auth header, successful result decoding and non-retried 401 handling. This verifies client transport, not any production OCR service or its recognition quality.

Raw lines 57 to 96
97## File conversion
99| Input | Examples | Requirement |
100|---|---|---|
101| PDF | `.pdf` | Native PDFium |
102| Word | `.doc`, `.docx`, `.docm`, `.odt`, `.rtf`, `.pages` | LibreOffice |
103| Presentations | `.ppt`, `.pptx`, `.pptm`, `.odp`, `.key` | LibreOffice |
104| Spreadsheets | `.xls`, `.xlsx`, `.xlsm`, `.xlsb`, `.ods`, `.csv`, `.tsv`, `.numbers` | LibreOffice |
105| Images | `.jpg`, `.jpeg`, `.png`, `.gif`, `.bmp`, `.tiff`, `.tif`, `.webp`, `.svg` | Built-in Rust image/resvg/usvg conversion |
107Office and image inputs convert to PDF first. **ImageMagick is not required in
1082.15.0**, despite stale internal enum names in source. Plain `.txt`/`.md` are not
109supported page-layout inputs and cannot be screenshot-rendered. A supported
110extension is not a promise that every file or animation/frame is preserved;
111check page count and rendering, especially for multi-frame images and Office formats.
113LibreOffice must be discoverable by the platform. Typical installation commands
114(illustrative; not run as part of this review):
116```bash
117# macOS
118brew install --cask libreoffice
119# Ubuntu/Debian
120sudo apt-get install libreoffice
121# Windows / Chocolatey
122choco install libreoffice-fresh
123```
125On Windows, put LibreOffice's `program` directory on PATH. Font availability,
126print areas, pagination and formula caches affect converted output. Validate
127scientific symbols and sheet coverage visually rather than equating a successful
128conversion with complete extraction.
130Sources reviewed 2026-10-01:
132- [OCR API specification](https://github.com/run-llama/liteparse/blob/d3a79177b9e9e570f8c2d9878ace601445fdaf76/OCR_API_SPEC.md)
133- [HTTP client](https://github.com/run-llama/liteparse/blob/d3a79177b9e9e570f8c2d9878ace601445fdaf76/crates/liteparse/src/ocr/http_simple.rs)
134- [Language downloads](https://github.com/run-llama/liteparse/blob/d3a79177b9e9e570f8c2d9878ace601445fdaf76/crates/liteparse/src/ocr/tesseract.rs)
135- [Conversion implementation](https://github.com/run-llama/liteparse/blob/d3a79177b9e9e570f8c2d9878ace601445fdaf76/crates/liteparse/src/conversion.rs)

File conversion

InputExamplesRequirement
PDF.pdfNative PDFium
Word.doc, .docx, .docm, .odt, .rtf, .pagesLibreOffice
Presentations.ppt, .pptx, .pptm, .odp, .keyLibreOffice
Spreadsheets.xls, .xlsx, .xlsm, .xlsb, .ods, .csv, .tsv, .numbersLibreOffice
Images.jpg, .jpeg, .png, .gif, .bmp, .tiff, .tif, .webp, .svgBuilt-in Rust image/resvg/usvg conversion

Office and image inputs convert to PDF first. ImageMagick is not required in 2.15.0, despite stale internal enum names in source. Plain .txt/.md are not supported page-layout inputs and cannot be screenshot-rendered. A supported extension is not a promise that every file or animation/frame is preserved; check page count and rendering, especially for multi-frame images and Office formats.

LibreOffice must be discoverable by the platform. Typical installation commands (illustrative; not run as part of this review):

# macOS
brew install --cask libreoffice
# Ubuntu/Debian
sudo apt-get install libreoffice
# Windows / Chocolatey
choco install libreoffice-fresh

On Windows, put LibreOffice's program directory on PATH. Font availability, print areas, pagination and formula caches affect converted output. Validate scientific symbols and sheet coverage visually rather than equating a successful conversion with complete extraction.

Sources reviewed 2026-10-01:

Raw lines 97 to 135