workflows.md
309 lines · skills/markitdown/references/workflows.md at 5f2bc03
Most marks come from matching words in the file, so treat them as pointers.
Practical Workflows
All workflows target MarkItDown 0.1.8 and use .markdown, not the legacy .text_content alias.
1. One Trusted Local Document
from hashlib import sha256
from pathlib import Path
from markitdown import MarkItDown
source = Path("protocol.docx").resolve(strict=True)
output = Path("protocol.md")
result = MarkItDown().convert_local(source)
output.write_text(result.markdown, encoding="utf-8")
provenance = {
"source": source.name,
"sha256": sha256(source.read_bytes()).hexdigest(),
"title": result.title,
"output": str(output),
}
print(provenance)For very large files, compute the hash incrementally rather than loading the source again.
Raw lines 5 to 292. Binary Upload
Validate size and type before conversion. Use BytesIO plus explicit hints:
from io import BytesIO
from markitdown import MarkItDown, StreamInfo
def convert_pdf_upload(payload: bytes, filename: str) -> str:
max_bytes = 25 * 1024 * 1024
if len(payload) > max_bytes:
raise ValueError("PDF exceeds the 25 MiB conversion limit")
if not payload.startswith(b"%PDF-"):
raise ValueError("Payload does not have a PDF signature")
result = MarkItDown().convert_stream(
BytesIO(payload),
stream_info=StreamInfo(
extension=".pdf",
mimetype="application/pdf",
filename=filename,
),
)
return result.markdownSignature checks are only one validation layer; they do not make the parser sandboxed.
Raw lines 30 to 593. Batch Directory Conversion
python scripts/batch_convert.py inputs/ outputs/ \
--recursive \
--extensions .pdf .docx .pptx .xlsx .csv .html \
--manifest outputs/manifest.jsonProperties:
- Local paths only
- Symlinks skipped
- Deterministic path ordering
- Relative directories preserved
- Output name retains the source suffix, e.g.
paper.pdf.md - Existing output skipped unless
--overwrite - Plugins off unless
--plugins - Nonzero exit when a conversion fails
Retry failed files after installing the missing format extra:
uv pip install "markitdown[pdf,docx,pptx,xlsx]==0.1.8"
python scripts/batch_convert.py inputs/ outputs/ --recursive --overwriteUse a fresh output directory when comparing converter versions.
Raw lines 60 to 884. Literature Collection
Input naming convention:
Author_Year_Title.pdfConvert and build indexes:
python scripts/convert_literature.py papers/ literature/ \
--recursive \
--create-indexOrganize by inferred year:
python scripts/convert_literature.py papers/ literature/ \
--recursive \
--organize-by-year \
--create-indexEach output contains provenance front matter:
---
title: "Example title"
author: "Smith"
year: "2025"
source: "incoming/Smith_2025_Example_Title.pdf"
converted_at: "2026-10-01T17:00:00+00:00"
markitdown_version: "0.1.8"
---The helper also writes catalog.json and INDEX.md when --create-index is requested. Metadata inferred from a filename is a convenience, not authoritative bibliographic metadata.
5. RAG Ingestion
Recommended sequence:
- Validate and hash the source.
- Convert with a pinned package/plugin version.
- Keep source metadata separate from converted text.
- Check non-empty output and expected section markers.
- Sanitize active HTML and unsafe links if Markdown will be rendered.
- Chunk by semantic headings with overlap.
- Store source/hash/converter/page-or-section provenance with every chunk.
- Mark chunks as untrusted source content in the retrieval prompt.
- Retain the original source for verification.
Example conversion envelope:
from dataclasses import asdict, dataclass
from hashlib import sha256
from importlib.metadata import version
from pathlib import Path
from markitdown import MarkItDown
@dataclass(frozen=True)
class ConvertedDocument:
source_name: str
source_sha256: str
converter_version: str
title: str | None
markdown: str
def convert_for_ingestion(path: Path) -> ConvertedDocument:
path = path.resolve(strict=True)
digest = sha256(path.read_bytes()).hexdigest()
result = MarkItDown().convert_local(path)
if not result.markdown.strip():
raise ValueError(f"Conversion produced no text: {path.name}")
return ConvertedDocument(
source_name=path.name,
source_sha256=digest,
converter_version=version("markitdown"),
title=result.title,
markdown=result.markdown,
)Do not treat Markdown line numbers as stable page citations. MarkItDown does not expose PDF page coordinates.
Raw lines 129 to 180Pass A: inventory
- Count files by extension and byte size.
- Detect duplicates by content hash.
- Exclude temporary, lock, and hidden files.
- Identify encrypted/unsupported inputs.
- Classify which files require external OCR/cloud processing.
Pass B: convert
- Use local built-ins first.
- Route only failed scanned/complex files to an approved OCR/cloud path.
- Record the selected mode per file.
- Compare file count, failure count, and output count.
This minimizes cost and external data transfer.
Raw lines 193 to 2017. Spreadsheet Orientation, Then Structured Analysis
Use MarkItDown to understand workbook organization:
from markitdown import MarkItDown
preview = MarkItDown().convert_local("assay-results.xlsx").markdown
print(preview)Then use a spreadsheet/dataframe library for calculations:
- Inspect formulas and cached values explicitly.
- Select the authoritative sheet/range.
- Preserve data types.
- Validate merged cells and hidden rows.
Do not parse a scientific numeric dataset back out of Markdown when the original workbook is available.
Raw lines 202 to 2218. Presentation Review
from markitdown import MarkItDown
result = MarkItDown().convert_local("lab-meeting.pptx")Review:
- Slide boundaries
- Title hierarchy
- Speaker notes
- Text in charts/images
- Claims encoded only by color/position
If images carry essential meaning, add an approved vision description or OCR pass and label generated text as model-derived.
Raw lines 222 to 2399. Scanned-PDF Escalation
- Try built-in PDF extraction.
- Detect unexpectedly short/empty output relative to page/file size.
- Confirm the source is scanned by visual inspection.
- Choose:
- Local OCR/layout parser for restricted data
markitdown-ocrfor approved vision-provider processing- Document Intelligence for Azure layout/OCR
- Content Understanding for structured fields/multimodal processing
- Validate a page sample and record the selected service/model/analyzer.
Do not automatically send failed local conversions to a cloud service.
Raw lines 240 to 25310. Application-Controlled Remote Resource
The application—not MarkItDown—should:
- Validate destination and every redirect
- Enforce time and byte limits
- Restrict MIME types
- Apply authentication safely
- Produce a bounded
requests.Response
Then:
result = MarkItDown().convert_response(validated_response)See security.md for the complete SSRF policy. A simple scheme check is not sufficient.
11. Deterministic Regression Corpus
Keep a small, redistributable corpus covering:
- Text-native and scanned PDFs
- DOCX headings/tables/equations
- PPTX notes/grouped shapes
- XLSX multiple sheets/merged cells
- CSV quoted fields and Unicode
- HTML links/lists/tables
- EPUB chapters
- JPEG/PNG metadata
For each fixture, assert:
- Conversion succeeds or fails with the expected category.
- Required headings/sentinel text are present.
- No source is unexpectedly routed to network/cloud.
- Output is UTF-8 and below a reasonable size.
- Package/plugin versions are recorded.
Avoid asserting a full byte-for-byte Markdown snapshot unless exact formatting stability is required; semantic assertions are less brittle.
Raw lines 272 to 29412. Conversion Quality Report
Track per file:
- Source identifier and hash
- Bytes and extension
- Converter mode
- MarkItDown/plugin version
- Output characters/lines
- Optional title
- Warning/failure category
- Manual validation status
- External provider/analyzer, if any
The bundled batch manifest provides a baseline for these records.
Raw lines 295 to 309