Skill Tracer
Skills › Scientific Agent Skills › pysam › migration_to_0_24.md

migration_to_0_24.md

192 lines · skills/pysam/references/migration_to_0_24.md at 5f2bc03

Open on GitHub
1# Migration to Pysam 0.24
2
3Use this checklist when moving an existing environment or pipeline to
4pysam 0.24.1.
5

Migration to Pysam 0.24

Use this checklist when moving an existing environment or pipeline to pysam 0.24.1.

Raw lines 1 to 5
6## Runtime Baseline
7
8- Pysam 0.24.1 was released 7 September 2026.
9- It was tested upstream with Python 3.9–3.15 (RC2).
10- It wraps HTSlib/samtools/bcftools 1.24.
11- It requires Cython 3 when building/extending from source.
12- Python 3.8 support ended with 0.24.0; 0.24.1 requires Python >=3.9.
14For a reproducible environment:
16```bash
17uv pip install "pysam==0.24.1"
18```
20If maintaining Python/Cython extensions that cimport pysam declarations,
21rebuild them against 0.24. Do not reuse binaries built against a different
22pysam ABI without verification.

Runtime Baseline

  • Pysam 0.24.1 was released 7 September 2026.
  • It was tested upstream with Python 3.9–3.15 (RC2).
  • It wraps HTSlib/samtools/bcftools 1.24.
  • It requires Cython 3 when building/extending from source.
  • Python 3.8 support ended with 0.24.0; 0.24.1 requires Python >=3.9.

For a reproducible environment:

uv pip install "pysam==0.24.1"

If maintaining Python/Cython extensions that cimport pysam declarations, rebuild them against 0.24. Do not reuse binaries built against a different pysam ABI without verification.

Raw lines 6 to 23
24## Changes Since 0.24.0
26- Embedded tools advance from 1.23.1 to 1.24; test command results again.
27- `tabix_compress` now raises if its input cannot be opened.
28- `reverse_complement` and `reverse_complement_inplace` replace manual tables.
29- FORMAT decoding/assignment supports P/LA/LR/LG/M cardinalities; preserve
30 versioned header and local-allele metadata.
31- Linux wheels restore S3 support on Debian/Ubuntu and dynamically load
32 networking libraries. Verify the libraries and actual protocol at runtime.
33- Wheel distributions no longer carry C/Cython source files or debug symbols.
35The existing 0.24.0 operational changes below remain relevant. See
36`sources.md` for upstream docstring/implementation discrepancies verified in
370.24.1 (notably variant fetch, pileup counts, BCF modes and tabix threads).

Changes Since 0.24.0

  • Embedded tools advance from 1.23.1 to 1.24; test command results again.
  • tabix_compress now raises if its input cannot be opened.
  • reverse_complement and reverse_complement_inplace replace manual tables.
  • FORMAT decoding/assignment supports P/LA/LR/LG/M cardinalities; preserve versioned header and local-allele metadata.
  • Linux wheels restore S3 support on Debian/Ubuntu and dynamically load networking libraries. Verify the libraries and actual protocol at runtime.
  • Wheel distributions no longer carry C/Cython source files or debug symbols.

The existing 0.24.0 operational changes below remain relevant. See sources.md for upstream docstring/implementation discrepancies verified in 0.24.1 (notably variant fetch, pileup counts, BCF modes and tabix threads).

Raw lines 24 to 38
39## CRAM Behavior

CRAM Behavior

Raw lines 39 to 40
41### Output version
43New CRAM output defaults to CRAM 3.1. If downstream software only supports
443.0:
46```python
47pysam.AlignmentFile(
48 "output.cram",
49 "wc",
50 header=header,
51 reference_filename="reference.fa",
52 format_options=["version=3.0"],
54```
56Upgrade downstream tooling instead of forcing 3.0 when possible.

Output version

New CRAM output defaults to CRAM 3.1. If downstream software only supports 3.0:

pysam.AlignmentFile(
    "output.cram",
    "wc",
    header=header,
    reference_filename="reference.fa",
    format_options=["version=3.0"],
)

Upgrade downstream tooling instead of forcing 3.0 when possible.

Raw lines 41 to 57
58### Reference lookup
60Implicit EBI reference fetching is no longer enabled by default. Existing
61pipelines that relied on hidden network retrieval can now fail.
63Preferred fix:
65```python
66pysam.AlignmentFile(
67 "input.cram",
68 "rc",
69 reference_filename="reference.fa",
71```
73Only configure `REF_PATH` and `REF_CACHE` when managed checksum-based lookup is
74intentional. Do not restore remote fetching without documenting network,
75cache, and provenance behavior.

Reference lookup

Implicit EBI reference fetching is no longer enabled by default. Existing pipelines that relied on hidden network retrieval can now fail.

Preferred fix:

pysam.AlignmentFile(
    "input.cram",
    "rc",
    reference_filename="reference.fa",
)

Only configure REF_PATH and REF_CACHE when managed checksum-based lookup is intentional. Do not restore remote fetching without documenting network, cache, and provenance behavior.

Raw lines 58 to 76
77## `format_options`
79Pysam 0.24 fixes Python 3 handling so `AlignmentFile(format_options=...)` and
80`HTSFile.add_hts_options()` accept documented `str` values rather than
81requiring bytes:
83```python
84format_options=["version=3.0"]
85```
87Remove workarounds that encode these strings to bytes.

format_options

Pysam 0.24 fixes Python 3 handling so AlignmentFile(format_options=...) and HTSFile.add_hts_options() accept documented str values rather than requiring bytes:

format_options=["version=3.0"]

Remove workarounds that encode these strings to bytes.

Raw lines 77 to 88
89## CIGAR Constants
91Pysam 0.23.2 restored top-level aliases such as `pysam.CMATCH` after a Cython
92behavior change, but the release notes warn that a future release will remove
93them.
95Migrate new and maintained code:
97```python
98# Compatibility alias
99pysam.CMATCH
101# Preferred
102pysam.CIGAR_OPS.CMATCH
103```
105Audit all CIGAR operators, not only `CMATCH`.

CIGAR Constants

Pysam 0.23.2 restored top-level aliases such as pysam.CMATCH after a Cython behavior change, but the release notes warn that a future release will remove them.

Migrate new and maintained code:

# Compatibility alias
pysam.CMATCH

# Preferred
pysam.CIGAR_OPS.CMATCH

Audit all CIGAR operators, not only CMATCH.

Raw lines 89 to 106
107## Modified Bases
109Pysam 0.24:
111- removes the prior five-modification-type limit in
112 `AlignedSegment.modified_bases`
113- fixes a crash for degenerate MM fields that describe no modified bases
115Retest MM/ML parsing with:
117- multiple modification codes
118- empty/degenerate MM tags
119- missing ML tags
120- forward and reverse alignments
121- `modified_bases_forward`

Modified Bases

Pysam 0.24:

  • removes the prior five-modification-type limit in AlignedSegment.modified_bases
  • fixes a crash for degenerate MM fields that describe no modified bases

Retest MM/ML parsing with:

  • multiple modification codes
  • empty/degenerate MM tags
  • missing ML tags
  • forward and reverse alignments
  • modified_bases_forward
Raw lines 107 to 122
123## Quality Conversion
125`pysam.array_to_qualitystring()` was optimized substantially in 0.24. Remove
126custom micro-optimizations only after confirming identical handling of missing
127or invalid values.

Quality Conversion

pysam.array_to_qualitystring() was optimized substantially in 0.24. Remove custom micro-optimizations only after confirming identical handling of missing or invalid values.

Raw lines 123 to 128
129## Coverage and Python 3.13+
131The 0.24 release fixes coverage functionality for Python 3.13 and later.
132Re-run `count_coverage()` regression cases when upgrading both Python and
133pysam:
135- zero-depth positions
136- base-quality threshold boundaries
137- duplicate/secondary/supplementary filtering
138- ambiguous query bases
139- contig boundaries

Coverage and Python 3.13+

The 0.24 release fixes coverage functionality for Python 3.13 and later. Re-run count_coverage() regression cases when upgrading both Python and pysam:

  • zero-depth positions
  • base-quality threshold boundaries
  • duplicate/secondary/supplementary filtering
  • ambiguous query bases
  • contig boundaries
Raw lines 129 to 140
141## Variant Documentation and Errors
143Pysam 0.24 adds documentation for many `VariantHeader` and `VariantRecord`
144internal field wrappers. Prefer documented public mappings and methods over
145introspection into Cython implementation details.
147Recent 0.23.x releases also improved `AlignmentFile` and `VariantFile` I/O
148exception handling. Catch specific `OSError`/`ValueError` conditions and do
149not depend on old message text.

Variant Documentation and Errors

Pysam 0.24 adds documentation for many VariantHeader and VariantRecord internal field wrappers. Prefer documented public mappings and methods over introspection into Cython implementation details.

Recent 0.23.x releases also improved AlignmentFile and VariantFile I/O exception handling. Catch specific OSError/ValueError conditions and do not depend on old message text.

Raw lines 141 to 150
151## Avoid Pysam 0.23.1 for Cython Extensions
153Pysam 0.23.1 was yanked because it broke binary compatibility for Cython
154projects by changing `AlignedSegment` size. Pysam 0.23.2 restored
155compatibility. Pure Python users were not affected in the same way.
157When migrating from 0.23:
159- skip 0.23.1
160- rebuild extension modules
161- test Cython cimports against the 0.24 declarations

Avoid Pysam 0.23.1 for Cython Extensions

Pysam 0.23.1 was yanked because it broke binary compatibility for Cython projects by changing AlignedSegment size. Pysam 0.23.2 restored compatibility. Pure Python users were not affected in the same way.

When migrating from 0.23:

  • skip 0.23.1
  • rebuild extension modules
  • test Cython cimports against the 0.24 declarations
Raw lines 151 to 162
163## Dispatcher Output
165Keep wrapped command output out of memory for bulk or binary operations:
167```python
168import pysam.samtools
170pysam.samtools.sort(
171 "-o",
172 "sorted.bam",
173 "input.bam",
174 catch_stdout=False,
176```
178This is not new to 0.24, but it is important when updating older examples that
179omit `catch_stdout=False`.

Dispatcher Output

Keep wrapped command output out of memory for bulk or binary operations:

import pysam.samtools

pysam.samtools.sort(
    "-o",
    "sorted.bam",
    "input.bam",
    catch_stdout=False,
)

This is not new to 0.24, but it is important when updating older examples that omit catch_stdout=False.

Raw lines 163 to 180
181## Regression Checklist
1831. Print and record `pysam.__version__` and `pysam.__samtools_version__`.
1842. Open representative BAM, CRAM, VCF.gz, BCF, FASTA, FASTQ, and tabix files.
1853. Compare numeric queries with equivalent region strings.
1864. Decode CRAM with no network and an explicit reference.
1875. Write/read CRAM 3.1; test 3.0 only if required.
1886. Rebuild all Cython extensions.
1897. Exercise pileup and `count_coverage()` thresholds.
1908. Test modified-base MM/ML edge cases.
1919. Run sort/index/normalization dispatchers with file output.
19210. Recreate indexes and compare known regions and aggregate counts.

Regression Checklist

  1. Print and record pysam.__version__ and pysam.__samtools_version__.
  2. Open representative BAM, CRAM, VCF.gz, BCF, FASTA, FASTQ, and tabix files.
  3. Compare numeric queries with equivalent region strings.
  4. Decode CRAM with no network and an explicit reference.
  5. Write/read CRAM 3.1; test 3.0 only if required.
  6. Rebuild all Cython extensions.
  7. Exercise pileup and count_coverage() thresholds.
  8. Test modified-base MM/ML edge cases.
  9. Run sort/index/normalization dispatchers with file output.
  10. Recreate indexes and compare known regions and aggregate counts.
Raw lines 181 to 192