Skill Tracer
Skills › Scientific Agent Skills › genomic-coordinates › format-conventions.md

format-conventions.md

238 lines · skills/genomic-coordinates/references/format-conventions.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Coordinate conventions, format by format
2
3Two independent choices define a convention, and formats mix them freely:
4
5- **Base**: is the first base of a contig called 0 or 1?
6- **Closure**: is the end coordinate part of the interval (inclusive) or one past
7 it (half-open)?
8
9There is no correlation between a format's age, its authorship, or its purpose and
10which pair it picked. UCSC alone ships both.

Coordinate conventions, format by format

Two independent choices define a convention, and formats mix them freely:

  • Base: is the first base of a contig called 0 or 1?
  • Closure: is the end coordinate part of the interval (inclusive) or one past it (half-open)?

There is no correlation between a format's age, its authorship, or its purpose and which pair it picked. UCSC alone ships both.

Raw lines 1 to 11
12## The table
14| Format | Convention | Length | Notes |
15| --- | --- | --- | --- |
16| BED (3/6/12) | 0-based half-open | `end - start` | `chromStart` may be 0 |
17| bedGraph | 0-based half-open | `end - start` | despite sitting next to WIG |
18| bigWig / bigBed | 0-based half-open | `end - start` | binary; matches BED |
19| narrowPeak / broadPeak | 0-based half-open | `end - start` | BED6+4 and BED6+3 |
20| WIG (fixedStep, variableStep) | 1-based inclusive | `end - start + 1` | the trap next to bedGraph |
21| GFF3 | 1-based inclusive | `end - start + 1` | `start <= end` always |
22| GTF / GFF2 | 1-based inclusive | `end - start + 1` | GENCODE, Ensembl |
23| VCF text | 1-based POS | `len(REF)` for literal REF span | event span can differ; BCF binary POS is 0-based |
24| SAM (text) | 1-based inclusive | from CIGAR | `POS` is the leftmost mapped base |
25| BAM (binary) | 0-based | from CIGAR | encoded alignment start |
26| CRAM | 1-based absolute alignment start | from alignment | AP can be delta encoded; do not treat it as BAM |
27| genePred / refFlat | 0-based half-open | `end - start` | `exonEnds` are exclusive |
28| PSL (BLAT) | 0-based half-open | `end - start` | see the minus-strand note below |
29| Picard interval_list | 1-based inclusive | `end - start + 1` | GATK targets, bait sets |
30| MAF — Mutation Annotation | 1-based inclusive | `End - Start + 1` | TCGA somatic calls |
31| MAF — Multiple Alignment | 0-based half-open | `size` field | UCSC whole-genome alignments |
32| samtools / tabix region string | 1-based inclusive | `end - start + 1` | `chr3:1000-2000` is 1001 bp |
33| UCSC browser position box | 1-based inclusive | `end - start + 1` | 1-based UI over 0-based files |
34| Ensembl REST region string | 1-based inclusive | `end - start + 1` | `chr:start..end:strand` |
35| IGV locus box | 1-based inclusive | `end - start + 1` | matches the UCSC box |
36| Bioconductor GRanges / IRanges | 1-based inclusive | `width()` | R ecosystem default |
37| PyRanges / pybedtools | 0-based half-open | `End - Start` | Python ecosystem default |
39`scripts/convert_coords.py --list` prints its supported convention names. It
40converts explicit interval triples; its limited file readers are listed in
41`../SKILL.md`. It is not a native parser for every format in this table.

The table

FormatConventionLengthNotes
BED (3/6/12)0-based half-openend - startchromStart may be 0
bedGraph0-based half-openend - startdespite sitting next to WIG
bigWig / bigBed0-based half-openend - startbinary; matches BED
narrowPeak / broadPeak0-based half-openend - startBED6+4 and BED6+3
WIG (fixedStep, variableStep)1-based inclusiveend - start + 1the trap next to bedGraph
GFF31-based inclusiveend - start + 1start <= end always
GTF / GFF21-based inclusiveend - start + 1GENCODE, Ensembl
VCF text1-based POSlen(REF) for literal REF spanevent span can differ; BCF binary POS is 0-based
SAM (text)1-based inclusivefrom CIGARPOS is the leftmost mapped base
BAM (binary)0-basedfrom CIGARencoded alignment start
CRAM1-based absolute alignment startfrom alignmentAP can be delta encoded; do not treat it as BAM
genePred / refFlat0-based half-openend - startexonEnds are exclusive
PSL (BLAT)0-based half-openend - startsee the minus-strand note below
Picard interval_list1-based inclusiveend - start + 1GATK targets, bait sets
MAF — Mutation Annotation1-based inclusiveEnd - Start + 1TCGA somatic calls
MAF — Multiple Alignment0-based half-opensize fieldUCSC whole-genome alignments
samtools / tabix region string1-based inclusiveend - start + 1chr3:1000-2000 is 1001 bp
UCSC browser position box1-based inclusiveend - start + 11-based UI over 0-based files
Ensembl REST region string1-based inclusiveend - start + 1chr:start..end:strand
IGV locus box1-based inclusiveend - start + 1matches the UCSC box
Bioconductor GRanges / IRanges1-based inclusivewidth()R ecosystem default
PyRanges / pybedtools0-based half-openEnd - StartPython ecosystem default

scripts/convert_coords.py --list prints its supported convention names. It converts explicit interval triples; its limited file readers are listed in ../SKILL.md. It is not a native parser for every format in this table.

Raw lines 12 to 42
Runs code scripts/convert_coords.py :39
43## The conversions worth memorising
45For ordinary nonempty spans, these two transformations cover the base/closure change:
47```
481-based inclusive -> 0-based half-open : start - 1, end
490-based half-open -> 1-based inclusive : start + 1, end
50```
52For these nonempty spans on one reference, only the start changes. This rule
53does not describe strand reversal, insertions, circular features, or liftover.

The conversions worth memorising

For ordinary nonempty spans, these two transformations cover the base/closure change:

1-based inclusive  ->  0-based half-open :  start - 1,  end
0-based half-open  ->  1-based inclusive :  start + 1,  end

For these nonempty spans on one reference, only the start changes. This rule does not describe strand reversal, insertions, circular features, or liftover.

Raw lines 43 to 54
55## Per-format detail

Per-format detail

Raw lines 55 to 56
57### BED
59`chromStart` is 0-based, `chromEnd` is exclusive. The first base of a chromosome
60is `0 1`. A single base at 1-based position 100 is `99 100`.
62`chromStart == chromEnd` is a **legal zero-length feature** — an insertion point
63between two bases, used by some variant tracks. Converting it requires feature
64semantics. GFF3 represents an insertion after base N with start=end=N; the same
65numbers can also denote a one-base feature. The generic converter reports
66`unrepresentable` for inclusive targets instead of guessing the feature meaning.
68BED12 block fields have exact rules that hand-written files routinely break:
70- `blockStarts` are offsets **from `chromStart`**, not absolute coordinates.
71- `blockStarts[0]` must be `0`.
72- `chromStart + blockStarts[-1] + blockSizes[-1]` must equal `chromEnd`.
73- `blockCount` must equal the length of both lists.
75`thickStart`/`thickEnd` delimit the CDS and must lie within `chromStart`/`chromEnd`;
76`thickStart == thickEnd` marks a non-coding transcript.
78narrowPeak's tenth column, `peak`, is an offset **from `chromStart`**, or `-1` when
79no summit was called. Adding it to `chromStart` gives the summit; treating it as an
80absolute coordinate puts the summit on the wrong chromosome arm.

BED

chromStart is 0-based, chromEnd is exclusive. The first base of a chromosome is 0 1. A single base at 1-based position 100 is 99 100.

chromStart == chromEnd is a legal zero-length feature — an insertion point between two bases, used by some variant tracks. Converting it requires feature semantics. GFF3 represents an insertion after base N with start=end=N; the same numbers can also denote a one-base feature. The generic converter reports unrepresentable for inclusive targets instead of guessing the feature meaning.

BED12 block fields have exact rules that hand-written files routinely break:

  • blockStarts are offsets **from chromStart**, not absolute coordinates.
  • blockStarts[0] must be 0.
  • chromStart + blockStarts[-1] + blockSizes[-1] must equal chromEnd.
  • blockCount must equal the length of both lists.

thickStart/thickEnd delimit the CDS and must lie within chromStart/chromEnd; thickStart == thickEnd marks a non-coding transcript.

narrowPeak's tenth column, peak, is an offset **from chromStart**, or -1 when no summit was called. Adding it to chromStart gives the summit; treating it as an absolute coordinate puts the summit on the wrong chromosome arm.

Raw lines 57 to 81
82### GFF3 and GTF
84Both are 1-based inclusive across nine tab-separated columns. `start <= end` is
85required **regardless of strand** — a minus-strand exon is still written with the
86smaller coordinate first, and orientation lives only in column 7. A GFF file with
87`start > end` is corrupt, not reverse-stranded.
89`start == 0` violates the format; inspect provenance before deciding that an
90entire file needs a shift. GFF3 circular features may have end larger than the
91landmark length to encode crossing its origin. The `##FASTA` directive terminates
92feature rows and introduces an optional sequence section.
94Column 8 is **phase** in GFF3 and **frame** in GTF, and they mean the same thing:
95the bases to skip from the transcriptional start of this segment to find the
96next complete codon. Retain those bases when concatenating CDS segments: they
97complete the codon begun in the preceding exon. Values are `0`, `1`, `2`, or `.`. It is not the reading
98frame of the feature's start position, and it is not `start % 3`. Every CDS
99feature should declare it; a missing phase requires annotation-specific review.
101Attribute syntax differs and parsers key on it:
103```
104GFF3 ID=exon1;Parent=transcript1;gene_name=TP53
105GTF gene_id "ENSG00000141510"; transcript_id "ENST00000269305";
106```
108A `.gtf` file containing GFF3 attributes may fail or lose attributes in readers
109that choose a parser from the extension.
111`exon_number` in GTF counts in **transcription order**, so on the minus strand
112exon 1 has the largest genomic coordinate. Sorting exons by coordinate and
113numbering them reproduces the right answer only on the plus strand.

GFF3 and GTF

Both are 1-based inclusive across nine tab-separated columns. start <= end is required regardless of strand — a minus-strand exon is still written with the smaller coordinate first, and orientation lives only in column 7. A GFF file with start > end is corrupt, not reverse-stranded.

start == 0 violates the format; inspect provenance before deciding that an entire file needs a shift. GFF3 circular features may have end larger than the landmark length to encode crossing its origin. The ##FASTA directive terminates feature rows and introduces an optional sequence section.

Column 8 is phase in GFF3 and frame in GTF, and they mean the same thing: the bases to skip from the transcriptional start of this segment to find the next complete codon. Retain those bases when concatenating CDS segments: they complete the codon begun in the preceding exon. Values are 0, 1, 2, or .. It is not the reading frame of the feature's start position, and it is not start % 3. Every CDS feature should declare it; a missing phase requires annotation-specific review.

Attribute syntax differs and parsers key on it:

GFF3   ID=exon1;Parent=transcript1;gene_name=TP53
GTF    gene_id "ENSG00000141510"; transcript_id "ENST00000269305";

A .gtf file containing GFF3 attributes may fail or lose attributes in readers that choose a parser from the extension.

exon_number in GTF counts in transcription order, so on the minus strand exon 1 has the largest genomic coordinate. Sorting exons by coordinate and numbering them reproduces the right answer only on the plus strand.

Raw lines 82 to 114
115### VCF
117`POS` is 1-based and refers to the first base of `REF`. The interval a record
118occupies is `POS` to `POS + len(REF) - 1`.
120For a simple indel, the unchanged padding base normally precedes the event;
121at the beginning of a contig, it follows it. Complex substitutions need not have
122an unchanged padding base.
124```
125reference ... A C G T T T A ...
126positions 4 5 6 7 8 9 10
128deletion of TT at 8-9 POS=7 REF=TTT ALT=T
129insertion of AA after 7 POS=7 REF=T ALT=TAA
130SNV at 7 POS=7 REF=T ALT=G
131```
133This illustrates padding before normalization; the deletion shifts left in its
134T repeat. `-` is not a VCF allele. `POS=0` and `POS=N+1` can occur in telomeric
135breakend records, whose REF/ALT need structural interpretation. `*` is a spanning
136deletion allele. Symbolic alleles and gVCF reference blocks cannot be reduced to
137`len(REF)`: VCF 4.5 uses allele-specific `SVLEN` and reference-block `FORMAT/LEN`,
138with legacy `INFO/END` retained for compatibility. Read the declared version.
140Allele representation has its own reference: `variant-representation.md`.

VCF

POS is 1-based and refers to the first base of REF. The interval a record occupies is POS to POS + len(REF) - 1.

For a simple indel, the unchanged padding base normally precedes the event; at the beginning of a contig, it follows it. Complex substitutions need not have an unchanged padding base.

reference   ...  A  C  G  T  T  T  A  ...
positions        4  5  6  7  8  9 10

deletion of TT at 8-9    POS=7  REF=TTT   ALT=T
insertion of AA after 7  POS=7  REF=T     ALT=TAA
SNV at 7                POS=7  REF=T     ALT=G

This illustrates padding before normalization; the deletion shifts left in its T repeat. - is not a VCF allele. POS=0 and POS=N+1 can occur in telomeric breakend records, whose REF/ALT need structural interpretation. * is a spanning deletion allele. Symbolic alleles and gVCF reference blocks cannot be reduced to len(REF): VCF 4.5 uses allele-specific SVLEN and reference-block FORMAT/LEN, with legacy INFO/END retained for compatibility. Read the declared version.

Allele representation has its own reference: variant-representation.md.

Raw lines 115 to 141
142### SAM, BAM, CRAM
144SAM text `POS` is 1-based; BAM stores a 0-based position. CRAM absolute
145alignment starts are 1-based and may be delta encoded. The library interface
146is a separate contract from the physical format:
148- `pysam`'s `AlignedSegment.reference_start` is **0-based**.
149- `pysam`'s `.pos` is the same 0-based number.
150- The `POS` you see in `samtools view` output is **1-based**.
152`reference_end` in pysam is 0-based exclusive, and is `None` for unmapped reads.
153`pysam.AlignmentFile.fetch(contig, start, end)` takes **0-based half-open**
154coordinates, but `fetch(region="chr1:100-200")` takes a **1-based inclusive**
155region string. The same method, two conventions, chosen by which argument you pass.

SAM, BAM, CRAM

SAM text POS is 1-based; BAM stores a 0-based position. CRAM absolute alignment starts are 1-based and may be delta encoded. The library interface is a separate contract from the physical format:

  • pysam's AlignedSegment.reference_start is 0-based.
  • pysam's .pos is the same 0-based number.
  • The POS you see in samtools view output is 1-based.

reference_end in pysam is 0-based exclusive, and is None for unmapped reads. pysam.AlignmentFile.fetch(contig, start, end) takes 0-based half-open coordinates, but fetch(region="chr1:100-200") takes a 1-based inclusive region string. The same method, two conventions, chosen by which argument you pass.

Raw lines 142 to 156
Names a skill pysam :148
157### Region strings
159`RNAME[:STARTPOS[-ENDPOS]]`, 1-based, both endpoints included, so `chr3:1000-2000`
160spans 1001 bases.
162Omitting the end does **not** mean a single base. `chr2:1000000` means position
1631,000,000 to the end of the chromosome. `scripts/convert_coords.py` refuses a
164region string without an explicit end rather than guessing which reading was meant.
166GRCh38 contig names can contain colons — `HLA-DRB1*12:17` is a real contig — so a
167region string is ambiguous without escaping. htslib resolves this with braces:
169```
170{HLA-DRB1*12:17} the whole contig
171{HLA-DRB1*12:17}:100-200 a region on it
172```
174Commas as thousands separators are accepted by htslib with
175`HTS_PARSE_THOUSANDS_SEP` and by the UCSC and IGV boxes, so `chr1:1,000,000-2,000,000`
176is valid input in most places and invalid in most file formats.

Region strings

RNAME[:STARTPOS[-ENDPOS]], 1-based, both endpoints included, so chr3:1000-2000 spans 1001 bases.

Omitting the end does not mean a single base. chr2:1000000 means position 1,000,000 to the end of the chromosome. scripts/convert_coords.py refuses a region string without an explicit end rather than guessing which reading was meant.

GRCh38 contig names can contain colons — HLA-DRB1*12:17 is a real contig — so a region string is ambiguous without escaping. htslib resolves this with braces:

{HLA-DRB1*12:17}          the whole contig
{HLA-DRB1*12:17}:100-200  a region on it

Commas as thousands separators are accepted by htslib with HTS_PARSE_THOUSANDS_SEP and by the UCSC and IGV boxes, so chr1:1,000,000-2,000,000 is valid input in most places and invalid in most file formats.

Raw lines 157 to 177
Runs code scripts/convert_coords.py :163
178### The UCSC split
180The UCSC Genome Browser displays and accepts 1-based inclusive coordinates in its
181position box, while the BED files it serves and consumes are 0-based half-open.
182Both are correct; they are different interfaces to the same data. A coordinate
183copied out of the browser window into a BED file is one base too far right.
185The UCSC Table Browser applies the same split per output format: BED output is
1860-based, "all fields from selected table" output of a genePred table is 0-based,
187and the position column shown in the browser is 1-based.

The UCSC split

The UCSC Genome Browser displays and accepts 1-based inclusive coordinates in its position box, while the BED files it serves and consumes are 0-based half-open. Both are correct; they are different interfaces to the same data. A coordinate copied out of the browser window into a BED file is one base too far right.

The UCSC Table Browser applies the same split per output format: BED output is 0-based, "all fields from selected table" output of a genePred table is 0-based, and the position column shown in the browser is 1-based.

Raw lines 178 to 188
189### PSL
191PSL uses 0-based half-open coordinates. For DNA query alignments, `qStart` and
192`qEnd` remain on the **forward query**, even when the query strand is minus.
193The per-block `qStarts` list instead uses reverse-complement coordinates on a
194minus query. Convert a reverse-coordinate block `[q, q + size)` to the forward
195query as `[qSize - q - size, qSize - q)`. Translated PSL may carry a second strand
196character for the target and uses different block-size units; use a PSL parser.

PSL

PSL uses 0-based half-open coordinates. For DNA query alignments, qStart and qEnd remain on the forward query, even when the query strand is minus. The per-block qStarts list instead uses reverse-complement coordinates on a minus query. Convert a reverse-coordinate block [q, q + size) to the forward query as [qSize - q - size, qSize - q). Translated PSL may carry a second strand character for the target and uses different block-size units; use a PSL parser.

Raw lines 189 to 197
198## Tool behaviour
200`bedtools` interprets recognized BED, GFF, and VCF inputs in their own
201conventions. Do not shift GFF columns and still label the result GFF. Output
202depends on the subcommand and options: `intersect -wa/-wb` preserves original
203records, and BAM input can produce BAM output. Do not assume all output is BED.
204Sorting is required for `-sorted` and indexing, not every bedtools operation.
206`bedtools slop` and `flank` require `-g` with chromosome sizes and clip to those
207bounds; omitting it is a usage error, not permission to emit negative starts.
209`rtracklayer::import()` converts BED coordinates to GRanges on read and export
210converts back. When manually constructing GRanges from BED numbers, adjust the
211start explicitly. Library accessors should be checked independently of file
212encodings (for example pysam `VariantRecord.pos` versus `.start`).

Tool behaviour

bedtools interprets recognized BED, GFF, and VCF inputs in their own conventions. Do not shift GFF columns and still label the result GFF. Output depends on the subcommand and options: intersect -wa/-wb preserves original records, and BAM input can produce BAM output. Do not assume all output is BED. Sorting is required for -sorted and indexing, not every bedtools operation.

bedtools slop and flank require -g with chromosome sizes and clip to those bounds; omitting it is a usage error, not permission to emit negative starts.

rtracklayer::import() converts BED coordinates to GRanges on read and export converts back. When manually constructing GRanges from BED numbers, adjust the start explicitly. Library accessors should be checked independently of file encodings (for example pysam VariantRecord.pos versus .start).

Raw lines 198 to 213
214## Official sources reviewed 2026-10-01
216- [UCSC format specifications](https://genome.ucsc.edu/FAQ/FAQformat.html): BED,
217 PSL, genePred, WIG, MAF and BED-derived formats.
218- [GENCODE GTF format](https://www.gencodegenes.org/pages/data_format.html).
219- [GFF3 specification](https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md):
220 insertion sites, circular landmarks, phase and embedded FASTA.
221- [VCF 4.5 and BCF](https://samtools.github.io/hts-specs/VCFv4.5.pdf),
222 [SAM/BAM](https://samtools.github.io/hts-specs/SAMv1.pdf),
223 [CRAM 3](https://samtools.github.io/hts-specs/CRAMv3.pdf).
224- [pysam API](https://pysam.readthedocs.io/en/latest/api.html).
225- [Ensembl sequence-region endpoint](https://rest.ensembl.org/documentation/info/sequence_region):
226 use `X:1000000..1000100:1` or `:-1`, not a `+`/`-` REST strand. The converter
227 accepts those signs as input convenience and emits numeric Ensembl strands.
228 Public GET, no key; at most 10 Mb per request, no pagination. A 101-base
229 plus/minus pair was fetched and reverse-complement checked during this review.
230- [bedtools intersect](https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html),
231 [slop](https://bedtools.readthedocs.io/en/latest/content/tools/slop.html),
232 [flank](https://bedtools.readthedocs.io/en/latest/content/tools/flank.html).
234- [GenomicRanges introduction](https://bioconductor.org/packages/release/bioc/vignettes/GenomicRanges/inst/doc/GenomicRangesIntroduction.html),
235 [pybedtools interval accessors](https://daler.github.io/pybedtools/intervals.html),
236 [PyRanges coordinate operations](https://pyranges1.readthedocs.io/en/latest/how_to_genomic_ops.html).
237 These library conventions were documentation-checked, not separately installed
238 for this standard-library helper suite.

Official sources reviewed 2026-10-01

Raw lines 214 to 238