Skill Tracer
Skills › Scientific Agent Skills › liteparse › output_formats.md

output_formats.md

104 lines · skills/liteparse/references/output_formats.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# LiteParse 2.15.0 Output Formats
2

LiteParse 2.15.0 Output Formats

Raw lines 1 to 2
3## Text and Markdown
4
5`lit parse document.pdf --format text` produces layout-preserved plain text;
6`--format markdown` reconstructs headings, lists, tables and links heuristically.
7Python `result.text` contains the selected document output, while each page has
8`text` and `markdown` fields. Text items remain available in every output mode.
9
10Markdown's default `image_mode="placeholder"` emits image references, not image
11files. Use `extract_images=True, image_output_dir="images"` (CLI
12`--extract-images --image-output-dir images`) to save bytes; `image_mode="off"`
13removes Markdown references. JSON carries image metadata, never image bytes.

Text and Markdown

lit parse document.pdf --format text produces layout-preserved plain text; --format markdown reconstructs headings, lists, tables and links heuristically. Python result.text contains the selected document output, while each page has text and markdown fields. Text items remain available in every output mode.

Markdown's default image_mode="placeholder" emits image references, not image files. Use extract_images=True, image_output_dir="images" (CLI --extract-images --image-output-dir images) to save bytes; image_mode="off" removes Markdown references. JSON carries image metadata, never image bytes.

Raw lines 3 to 14
15## Native CLI JSON is not the Python object schema
17```bash
18lit parse document.pdf --format json --no-ocr -o document.json
19```
21A minimal 2.15.0 native-text result has this shape (illustrative values):
23```json
25 "total_pages": 1,
26 "pages": [{
27 "page": 1,
28 "width": 612.0,
29 "height": 792.0,
30 "text": "Materials and Methods",
31 "text_items": [{
32 "text": "Materials and Methods",
33 "x": 72.0,
34 "y": 120.28,
35 "width": 242.76,
36 "height": 26.78,
37 "font_name": "Helvetica",
38 "font_size": 24.0,
39 "confidence": 1.0
40 }]
41 }]
43```
45There is **no top-level `text`** and the page-number key is **`page`**, not
46`page_num`. Optional fields may be absent. `total_pages` counts source pages
47before selection/capping, not `len(pages)`. When present, CLI `page_errors`
48entries use `page` and `message`. Python uses `PageError.page_number`.
50`--extract-blocks`, `--extract-annotations`, `--extract-form-fields`,
51`--extract-structure-tree`, `--extract-vector-graphics`, and `--complexity` add
52opt-in page fields. These can enlarge output substantially. `--extract-text-metadata`
53adds rotation and richer glyph/font metadata. Read the
54[released serializer](https://github.com/run-llama/liteparse/blob/d3a79177b9e9e570f8c2d9878ace601445fdaf76/crates/liteparse/src/output/json.rs)
55for the full schema. npm's CLI also uses snake_case; Node library objects use camelCase.

Native CLI JSON is not the Python object schema

lit parse document.pdf --format json --no-ocr -o document.json

A minimal 2.15.0 native-text result has this shape (illustrative values):

{
  "total_pages": 1,
  "pages": [{
    "page": 1,
    "width": 612.0,
    "height": 792.0,
    "text": "Materials and Methods",
    "text_items": [{
      "text": "Materials and Methods",
      "x": 72.0,
      "y": 120.28,
      "width": 242.76,
      "height": 26.78,
      "font_name": "Helvetica",
      "font_size": 24.0,
      "confidence": 1.0
    }]
  }]
}

There is **no top-level text and the page-number key is page**, not page_num. Optional fields may be absent. total_pages counts source pages before selection/capping, not len(pages). When present, CLI page_errors entries use page and message. Python uses PageError.page_number.

--extract-blocks, --extract-annotations, --extract-form-fields, --extract-structure-tree, --extract-vector-graphics, and --complexity add opt-in page fields. These can enlarge output substantially. --extract-text-metadata adds rotation and richer glyph/font metadata. Read the released serializer for the full schema. npm's CLI also uses snake_case; Node library objects use camelCase.

Raw lines 15 to 56
57## Python and the bundled batch wrapper
59Python exposes `result.text`, `result.total_pages`, `result.pages`,
60`result.page_errors`; page objects expose `page_num`, `text_items`, `text`,
61`markdown`, dimensions and optional metadata. `result.num_pages == len(result.pages)`.
62`result.get_page(1)` returns `None` if page 1 was not parsed.
64The bundled `scripts/batch_parse_dir.py` deliberately emits a compact custom
65JSON subset: `text`, `total_pages`, `page_errors`, and pages with `page_num`,
66`width`, `height`, `text`, `markdown`, `page_label`, and basic `text_items`.
67It excludes image bytes and optional rich metadata. Use native CLI JSON when
68that schema or advanced extraction fields are required. The wrapper rejects
69partial-page results, page-count truncation and existing output files, reports failures
70and exits nonzero. For more than the default 1000 pages, use the native CLI/API
71with an explicit `max_pages` and reconcile the returned pages.
72It mirrors input subdirectories and appends the output extension to the whole
73source name, e.g. `supplement/paper.pdf.json`.

Python and the bundled batch wrapper

Python exposes result.text, result.total_pages, result.pages, result.page_errors; page objects expose page_num, text_items, text, markdown, dimensions and optional metadata. result.num_pages == len(result.pages). result.get_page(1) returns None if page 1 was not parsed.

The bundled scripts/batch_parse_dir.py deliberately emits a compact custom JSON subset: text, total_pages, page_errors, and pages with page_num, width, height, text, markdown, page_label, and basic text_items. It excludes image bytes and optional rich metadata. Use native CLI JSON when that schema or advanced extraction fields are required. The wrapper rejects partial-page results, page-count truncation and existing output files, reports failures and exits nonzero. For more than the default 1000 pages, use the native CLI/API with an explicit max_pages and reconcile the returned pages. It mirrors input subdirectories and appends the output extension to the whole source name, e.g. supplement/paper.pdf.json.

Raw lines 57 to 74
Runs code scripts/batch_parse_dir.py :64
75## Boxes, confidence, and validation
77- Text-item boxes are `(x, y, width, height)` in top-left page viewport coordinates,
78 at 72 units per inch; right/down are positive. Store each page's dimensions.
79- For a screenshot of width `Wpx` and height `Hpx`, scale x/width by
80 `Wpx / page.width` and y/height by `Hpx / page.height`. Rotations/crops must use
81 the same parser settings. Never overlay raw PDF coordinates onto a 150-DPI image.
82- HTTP OCR returns `[x1, y1, x2, y2]` in uploaded-image pixels. LiteParse maps
83 these into page coordinates and merges native/OCR text.
84- A text item may be a multiword span; Python `emit_word_boxes=True` opts into
85 `item.words`. The default does not promise per-token boxes.
86- Python native-text `confidence` may be `None`; **CLI JSON substitutes 1.0**.
87 OCR scores are normalized to 0–1, but are not calibrated probabilities or a
88 guarantee of a correct scientific transcription. Confidence alone cannot
89 identify OCR provenance.
90- Compare requested page numbers with returned pages and inspect `page_errors`.
91 Review tables, units, equations and multi-column reading order against PNGs.
92 Keep low-confidence source evidence available for review rather than silently
93 discarding it from an allegedly complete corpus.
95```python
96from liteparse import search_items
98page = result.get_page(1)
99hits = search_items(page.text_items, "Supplementary Table 1") if page else []
100for hit in hits:
101 print(hit.text, hit.x, hit.y, hit.width, hit.height)
102```
104Reviewed 2026-10-01 against 2.15.0 source and synthetic PDF/PNG runtime fixtures.

Boxes, confidence, and validation

  • Text-item boxes are (x, y, width, height) in top-left page viewport coordinates, at 72 units per inch; right/down are positive. Store each page's dimensions.
  • For a screenshot of width Wpx and height Hpx, scale x/width by Wpx / page.width and y/height by Hpx / page.height. Rotations/crops must use the same parser settings. Never overlay raw PDF coordinates onto a 150-DPI image.
  • HTTP OCR returns [x1, y1, x2, y2] in uploaded-image pixels. LiteParse maps these into page coordinates and merges native/OCR text.
  • A text item may be a multiword span; Python emit_word_boxes=True opts into item.words. The default does not promise per-token boxes.
  • Python native-text confidence may be None; CLI JSON substitutes 1.0. OCR scores are normalized to 0–1, but are not calibrated probabilities or a guarantee of a correct scientific transcription. Confidence alone cannot identify OCR provenance.
  • Compare requested page numbers with returned pages and inspect page_errors. Review tables, units, equations and multi-column reading order against PNGs. Keep low-confidence source evidence available for review rather than silently discarding it from an allegedly complete corpus.
from liteparse import search_items

page = result.get_page(1)
hits = search_items(page.text_items, "Supplementary Table 1") if page else []
for hit in hits:
    print(hit.text, hit.x, hit.y, hit.width, hit.height)

Reviewed 2026-10-01 against 2.15.0 source and synthetic PDF/PNG runtime fixtures.

Raw lines 75 to 104