Skill Tracer
Skills › Scientific Agent Skills › gtars › coverage.md

coverage.md

225 lines · skills/gtars/references/coverage.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Coverage, uniwig, and bigWig
2
3Source-reviewed against `gtars-cli==0.10.0` / `gtars-uniwig==0.9.0` on
4**2026-10-01**; CLI/Rust templates were not executed in this audit. Published
5crate source takes precedence when older guide examples differ.
6

Coverage, uniwig, and bigWig

Source-reviewed against gtars-cli==0.10.0 / gtars-uniwig==0.9.0 on 2026-10-01; CLI/Rust templates were not executed in this audit. Published crate source takes precedence when older guide examples differ.

Raw lines 1 to 6
7## Distinguish two meanings of coverage
8
9- Python `RegionSet.coverage(other)` returns a single fraction of base pairs in
10 the first set covered by the second.
11- `gtars uniwig` creates positional signal tracks (WIG, NPY, bedGraph, bigWig,
12 and limited BAM-derived outputs).
14Python 0.10.0 does not export `gtars.uniwig`. Old examples using
15`gtars.uniwig.coverage_from_bed`, `coverage.normalize()`, `smooth()`,
16`call_peaks()`, or `to_bigwig()` are not current APIs.

Distinguish two meanings of coverage

  • Python RegionSet.coverage(other) returns a single fraction of base pairs in the first set covered by the second.
  • gtars uniwig creates positional signal tracks (WIG, NPY, bedGraph, bigWig, and limited BAM-derived outputs).

Python 0.10.0 does not export gtars.uniwig. Old examples using gtars.uniwig.coverage_from_bed, coverage.normalize(), smooth(), call_peaks(), or to_bigwig() are not current APIs.

Raw lines 7 to 17
18## Input contract
20For BED and narrowPeak:
221. use 0-based half-open intervals;
232. supply the exact assembly's local `chrom.sizes`;
243. require every contig to exist and every end to be within bounds;
254. sort by chromosome dictionary order, then numeric start/end;
265. use one local file (BED, narrowPeak, or BAM);
276. preserve strand separately—uniwig's BED path produces start/end/core counts,
28 not a generic BED6 strand-aware split.
30The official module guide states that uniwig expects a single chromosome-sorted
31input. Never concatenate samples, patients, or assemblies without a reviewed
32aggregation policy.
34Run the deterministic preflight:
36```bash
37python3 -B scripts/coverage_preflight.py \
38 --input fragments.sorted.bed.gz \
39 --input-type bed \
40 --chrom-sizes GRCh38.p14.chrom.sizes \
41 --assembly GRCh38.p14 \
42 --output-prefix derived/sample01 \
43 --output-type bw \
44 --count-type core \
45 --threads 4
46```
48It validates local paths, bounds, sorting, Gtars `u32` coordinates, output
49prefix collisions (including derived output files), and a dense-value budget
50rounded separately for each contig. It writes and executes
51nothing.

Input contract

For BED and narrowPeak:

  1. use 0-based half-open intervals;
  2. supply the exact assembly's local chrom.sizes;
  3. require every contig to exist and every end to be within bounds;
  4. sort by chromosome dictionary order, then numeric start/end;
  5. use one local file (BED, narrowPeak, or BAM);
  6. preserve strand separately—uniwig's BED path produces start/end/core counts, not a generic BED6 strand-aware split.

The official module guide states that uniwig expects a single chromosome-sorted input. Never concatenate samples, patients, or assemblies without a reviewed aggregation policy.

Run the deterministic preflight:

python3 -B scripts/coverage_preflight.py \
  --input fragments.sorted.bed.gz \
  --input-type bed \
  --chrom-sizes GRCh38.p14.chrom.sizes \
  --assembly GRCh38.p14 \
  --output-prefix derived/sample01 \
  --output-type bw \
  --count-type core \
  --threads 4

It validates local paths, bounds, sorting, Gtars u32 coordinates, output prefix collisions (including derived output files), and a dense-value budget rounded separately for each contig. It writes and executes nothing.

Raw lines 18 to 52
Runs code scripts/coverage_preflight.py :37
53## Current batch CLI
55BigWig generation uses the root `uniwig` command directly—there is no `generate`
56subcommand:
58```bash
59gtars uniwig \
60 --file fragments.sorted.bed.gz \
61 --filetype bed \
62 --chromref GRCh38.p14.chrom.sizes \
63 --smoothsize 5 \
64 --stepsize 1 \
65 --fileheader derived/sample01_ \
66 --outputtype bw \
67 --counttype core \
68 --threads 4 \
69 --zoom 1
70```
72Equivalent short options are `-f`, `-t`, `-c`, `-m`, `-s`, `-l`, `-y`, `-u`,
73`-p`, and `-z`. Valid batch count types are:
75- `start`: accumulations at interval starts;
76- `end`: accumulations at interval ends;
77- `core`: interval-body accumulations;
78- `all`: produce start, end, and core;
79- `shift`: BAM-specific shifted workflow.
81The implementation accepts `wig`, `npy`, `bedgraph`, `bw`, and `bigwig` strings
82along relevant paths, but use the documented compact `bw` for BigWig. BED and
83narrowPeak can produce WIG, NPY, bedGraph, or BigWig. BAM paths produce BigWig or
84BED in the documented workflow.
86Other batch flags:
88- `--score` uses narrowPeak score;
89- `--bamscale FLOAT` scales BAM values (default `1.0`);
90- `--no-bamshift` disables direction-aware BAM shifting;
91- `--wigstep fixed|variable` selects WIG step style;
92- `--debug` increases output.
94Validate scientific meaning before using start/end/shift signals. ATAC cut-site
95shifts and ChIP fragment-body counts are not interchangeable.

Current batch CLI

BigWig generation uses the root uniwig command directly—there is no generate subcommand:

gtars uniwig \
  --file fragments.sorted.bed.gz \
  --filetype bed \
  --chromref GRCh38.p14.chrom.sizes \
  --smoothsize 5 \
  --stepsize 1 \
  --fileheader derived/sample01_ \
  --outputtype bw \
  --counttype core \
  --threads 4 \
  --zoom 1

Equivalent short options are -f, -t, -c, -m, -s, -l, -y, -u, -p, and -z. Valid batch count types are:

  • start: accumulations at interval starts;
  • end: accumulations at interval ends;
  • core: interval-body accumulations;
  • all: produce start, end, and core;
  • shift: BAM-specific shifted workflow.

The implementation accepts wig, npy, bedgraph, bw, and bigwig strings along relevant paths, but use the documented compact bw for BigWig. BED and narrowPeak can produce WIG, NPY, bedGraph, or BigWig. BAM paths produce BigWig or BED in the documented workflow.

Other batch flags:

  • --score uses narrowPeak score;
  • --bamscale FLOAT scales BAM values (default 1.0);
  • --no-bamshift disables direction-aware BAM shifting;
  • --wigstep fixed|variable selects WIG step style;
  • --debug increases output.

Validate scientific meaning before using start/end/shift signals. ATAC cut-site shifts and ChIP fragment-body counts are not interchangeable.

Raw lines 53 to 96
97## Streaming mode
99For large **BED** input, CLI 0.10.0 exposes a streaming processor. Memory still
100depends on the operation and interval distribution; pilot representative data:
102```bash
103gtars uniwig \
104 --file fragments.sorted.bed.gz \
105 --filetype bed \
106 --chromref GRCh38.p14.chrom.sizes \
107 --smoothsize 5 \
108 --stepsize 1 \
109 --fileheader derived/sample01_ \
110 --outputtype bedgraph \
111 --counttype core \
112 --streaming \
113 --dense 0
114```
116Streaming constraints in tagged source:
118- only BED input;
119- only `wig` or `bedgraph` output, not BigWig or NPY;
120- count type `start`, `end`, `core`, or `all` (not BAM `shift`);
121- `--dense 0` is sparse, `--dense -1` is fully dense, and positive `N` fills
122 gaps no wider than `N`;
123- `--stdout` is available; multiple count types receive separator comments.
125If stdin is used with `--counttype all`, the handler buffers stdin into memory so
126it can replay it. Do not claim constant memory for that combination.

Streaming mode

For large BED input, CLI 0.10.0 exposes a streaming processor. Memory still depends on the operation and interval distribution; pilot representative data:

gtars uniwig \
  --file fragments.sorted.bed.gz \
  --filetype bed \
  --chromref GRCh38.p14.chrom.sizes \
  --smoothsize 5 \
  --stepsize 1 \
  --fileheader derived/sample01_ \
  --outputtype bedgraph \
  --counttype core \
  --streaming \
  --dense 0

Streaming constraints in tagged source:

  • only BED input;
  • only wig or bedgraph output, not BigWig or NPY;
  • count type start, end, core, or all (not BAM shift);
  • --dense 0 is sparse, --dense -1 is fully dense, and positive N fills gaps no wider than N;
  • --stdout is available; multiple count types receive separator comments.

If stdin is used with --counttype all, the handler buffers stdin into memory so it can replay it. Do not claim constant memory for that combination.

Raw lines 97 to 127
128## BAM QC and BAM coverage
130Library-complexity metrics are a subcommand:
132```bash
133gtars uniwig bamqc \
134 --input aligned.bam \
135 --output bamqc.tsv \
136 --threads 1
137```
139Parallel BAM QC (`--threads >1`) requires a `.bai` index. Bound BAM size, index
140size, decompression work, threads, and output. Metrics NRF/PBC1/PBC2 are technical
141QC summaries, not evidence of biological quality or suitability.
143For BAM-to-bigWig, the batch path requires the same `--smoothsize`,
144`--stepsize`, `--fileheader`, `--chromref`, and output controls. Keep alignment
145assembly, filtering, duplicate policy, paired-end handling, and shift/scaling in
146the provenance record.

BAM QC and BAM coverage

Library-complexity metrics are a subcommand:

gtars uniwig bamqc \
  --input aligned.bam \
  --output bamqc.tsv \
  --threads 1

Parallel BAM QC (--threads >1) requires a .bai index. Bound BAM size, index size, decompression work, threads, and output. Metrics NRF/PBC1/PBC2 are technical QC summaries, not evidence of biological quality or suitability.

For BAM-to-bigWig, the batch path requires the same --smoothsize, --stepsize, --fileheader, --chromref, and output controls. Keep alignment assembly, filtering, duplicate policy, paired-end handling, and shift/scaling in the provenance record.

Raw lines 128 to 147
148## BigWig preflight and postflight
150Before generation:
152- verify the exact chromosome dictionary and checksum;
153- reject unknown/out-of-bounds contigs;
154- ensure sorted input and numeric signal values;
155- reserve disk for intermediate bedGraph plus final BigWig;
156- set threads explicitly (upstream batch default is 6);
157- use a new output prefix;
158- avoid patient identifiers in filenames and track labels.
160After generation:
162- verify nonzero file size and BigWig readability with a trusted, pinned reader;
163- compare its chromosome dictionary and lengths with the input checksum;
164- query fixed synthetic positions with known expected coverage;
165- check min/max/NaN behavior and start/end/core suffixes;
166- record SHA-256, tool versions, parameters, and input hashes.
168UCSC documents bedGraph coordinates as 0-based half-open and numerically ordered.
169Its BigWig tools require matching chromosome sizes. A successful binary write
170does not prove the assembly or signal semantics are correct.

BigWig preflight and postflight

Before generation:

  • verify the exact chromosome dictionary and checksum;
  • reject unknown/out-of-bounds contigs;
  • ensure sorted input and numeric signal values;
  • reserve disk for intermediate bedGraph plus final BigWig;
  • set threads explicitly (upstream batch default is 6);
  • use a new output prefix;
  • avoid patient identifiers in filenames and track labels.

After generation:

  • verify nonzero file size and BigWig readability with a trusted, pinned reader;
  • compare its chromosome dictionary and lengths with the input checksum;
  • query fixed synthetic positions with known expected coverage;
  • check min/max/NaN behavior and start/end/core suffixes;
  • record SHA-256, tool versions, parameters, and input hashes.

UCSC documents bedGraph coordinates as 0-based half-open and numerically ordered. Its BigWig tools require matching chromosome sizes. A successful binary write does not prove the assembly or signal semantics are correct.

Raw lines 148 to 171
172## Rust APIs
174Enable only uniwig:
176```toml
177[dependencies]
178gtars = { version = "=0.10.0", default-features = false, features = ["uniwig"] }
179```
181The wrapper re-exports `gtars_uniwig` as `gtars::uniwig`. The primary batch
182function is:
184```text
185uniwig_main(
186 vec_count_type, smoothsize, filepath, chromsizerefpath, bwfileheader,
187 output_type, filetype, num_threads, score, stepsize, zoom, debug,
188 bam_shift, bam_scale, wigstep
189) -> Result<(), Box<dyn Error>>
190```
192It is deliberately string-heavy and has many arguments; prefer the pinned CLI
193unless embedding is necessary. The typed streaming API is:
195```text
196uniwig::stream::uniwig_streaming(
197 input, output, chrom_sizes, smooth_size, step_size,
198 CountType::{Start|End|Core},
199 OutputFormat::{Wig|BedGraph},
200 max_gap
202```
204`read_chrom_sizes(BufRead)` parses the dictionary. BigWig is a batch API, not a
205streaming `OutputFormat`.

Rust APIs

Enable only uniwig:

[dependencies]
gtars = { version = "=0.10.0", default-features = false, features = ["uniwig"] }

The wrapper re-exports gtars_uniwig as gtars::uniwig. The primary batch function is:

uniwig_main(
  vec_count_type, smoothsize, filepath, chromsizerefpath, bwfileheader,
  output_type, filetype, num_threads, score, stepsize, zoom, debug,
  bam_shift, bam_scale, wigstep
) -> Result<(), Box<dyn Error>>

It is deliberately string-heavy and has many arguments; prefer the pinned CLI unless embedding is necessary. The typed streaming API is:

uniwig::stream::uniwig_streaming(
  input, output, chrom_sizes, smooth_size, step_size,
  CountType::{Start|End|Core},
  OutputFormat::{Wig|BedGraph},
  max_gap
)

read_chrom_sizes(BufRead) parses the dictionary. BigWig is a batch API, not a streaming OutputFormat.

Raw lines 172 to 206
207## Threading and resources
209- Batch uniwig builds a Rayon pool of exactly `--threads`; the CLI default is 6.
210- Streaming mode is not controlled by the batch `--threads` path.
211- Output work can scale with total assembly span divided by step size, not only
212 with BED row count.
213- Smoothing, dense gap filling, three count types, BigWig intermediates, and high
214 thread counts can multiply memory/disk.
215- Start with one thread and one small synthetic contig. Increase only after
216 measuring peak RSS, temporary disk, throughput, and deterministic equivalence.

Threading and resources

  • Batch uniwig builds a Rayon pool of exactly --threads; the CLI default is 6.
  • Streaming mode is not controlled by the batch --threads path.
  • Output work can scale with total assembly span divided by step size, not only with BED row count.
  • Smoothing, dense gap filling, three count types, BigWig intermediates, and high thread counts can multiply memory/disk.
  • Start with one thread and one small synthetic contig. Increase only after measuring peak RSS, temporary disk, throughput, and deterministic equivalence.
Raw lines 207 to 217
218## Official sources (accessed 2026-10-01)
220- [Gtars uniwig module guide](https://docs.bedbase.org/gtars/uniwig/)
221- [CLI uniwig parser at gtars-v0.10.0](https://github.com/databio/gtars/blob/gtars-v0.10.0/gtars-cli/src/uniwig/cli.rs)
222- [CLI uniwig handler at gtars-v0.10.0](https://github.com/databio/gtars/blob/gtars-v0.10.0/gtars-cli/src/uniwig/handlers.rs)
223- [Rust uniwig 0.9.0 source](https://github.com/databio/gtars/tree/gtars-v0.10.0/gtars-uniwig)
224- [UCSC bedGraph format](https://genome.ucsc.edu/goldenPath/help/bedgraph.html)
225- [UCSC BigWig format](https://genome.ucsc.edu/goldenPath/help/bigWig.html)