Skill Tracer
Skills › Scientific Agent Skills › dnanexus-integration › sources.md

sources.md

206 lines · skills/dnanexus-integration/references/sources.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Sources and Version Baseline
2

Sources and Version Baseline

Raw lines 1 to 2
3## Verification
4
5Last verified: **2026-09-30**
6
7| Component | Verified baseline |
8|---|---|
9| DNAnexus Platform docs | 2026 documentation and release notes through 2026-09-22 |
10| dx-toolkit / dxpy | 0.415.0, released 2026-09-22 |
11| Python requirement | Python 3.8+ on PyPI |
12| App Execution Environment | Ubuntu 24.04 and 20.04, version `0` |
13| dxCompiler | 2.18.0 (released 2026-08-05) |
14| Upload Agent | 1.5.33 |
15| Download Agent | 0.6.4 |
16| dxFUSE | 1.6.1 |
17| Nextaur | 1.16.0 (toolkit release notes) |
18| Nextflow engines exposed by dx-toolkit 0.415.0 | 25.10 and 24.10 |
19| Nextflow 25.10 asset baseline | Nextflow 25.10.4 with nf-amazon 3.4.4 |
21The dxpy symbols/signatures, CLI help, and bundled offline helpers were executed
22locally against the pinned SDK. Platform contracts and the remaining release
23versions were reviewed in current official documentation/source. Transfers,
24builds, launches, and compiler/Nextflow execution examples are illustrative;
25no authenticated or billable end-to-end run was performed. The current
26dxCompiler release is established by its tagged release, because the `develop`
27changelog still labels the 0.6.4 download-agent change as unreleased.
29Version-specific examples in this skill are pinned for reproducibility. Before
30upgrading, inspect release notes, run `scripts/inspect_dxpy.py`, and rebuild/test
31apps in a non-production project.

Verification

Last verified: 2026-09-30

ComponentVerified baseline
DNAnexus Platform docs2026 documentation and release notes through 2026-09-22
dx-toolkit / dxpy0.415.0, released 2026-09-22
Python requirementPython 3.8+ on PyPI
App Execution EnvironmentUbuntu 24.04 and 20.04, version 0
dxCompiler2.18.0 (released 2026-08-05)
Upload Agent1.5.33
Download Agent0.6.4
dxFUSE1.6.1
Nextaur1.16.0 (toolkit release notes)
Nextflow engines exposed by dx-toolkit 0.415.025.10 and 24.10
Nextflow 25.10 asset baselineNextflow 25.10.4 with nf-amazon 3.4.4

The dxpy symbols/signatures, CLI help, and bundled offline helpers were executed locally against the pinned SDK. Platform contracts and the remaining release versions were reviewed in current official documentation/source. Transfers, builds, launches, and compiler/Nextflow execution examples are illustrative; no authenticated or billable end-to-end run was performed. The current dxCompiler release is established by its tagged release, because the develop changelog still labels the 0.6.4 download-agent change as unreleased.

Version-specific examples in this skill are pinned for reproducibility. Before upgrading, inspect release notes, run scripts/inspect_dxpy.py, and rebuild/test apps in a non-production project.

Raw lines 3 to 32
Runs code scripts/inspect_dxpy.py :30
33## Documentation Index and Releases
35- [DNAnexus documentation index (`llms.txt`)](https://documentation.dnanexus.com/llms.txt)
36- [DNAnexus 2026 release notes](https://documentation.dnanexus.com/release-notes.md)
37- [Downloads](https://documentation.dnanexus.com/downloads.md)
38- [Index of `dx` commands](https://documentation.dnanexus.com/user/helpstrings-of-sdk-command-line-utilities.md)
39- [dx-toolkit GitHub repository](https://github.com/dnanexus/dx-toolkit)
40- [dx-toolkit changelog](https://github.com/dnanexus/dx-toolkit/blob/master/CHANGELOG.md)
41- [dxpy on PyPI](https://pypi.org/project/dxpy/)
42- [Generated Python API reference](https://autodoc.dnanexus.com/bindings/python/current/)
44The release notes are date-versioned by platform deployment. The skill
45baseline incorporates:
47- 2026-09-22: dx-toolkit 415.0 Nextflow offline mode
48- 2026-09-15: CLI login default expiration aligned to 18 hours
49- 2026-08-18: normal login sessions shortened to two hours idle, 18 hours absolute
50- 2026-08-11: user API tokens blocked for UKB RAP; dxCompiler 2.18.0 / dxda 0.6.4
51- 2026-08-04: rebuild Nextflow applets to pick up upload-recovery fixes
52- 2026-07-21: dx-toolkit 410.0 malicious-file download warning
53- 2026-07-14: strengthened session/password controls
54- 2026-06-23: dxCompiler 2.17.0 authenticated WDL imports
55- 2026-06-09: file download API `securityStatus`
56- 2026-05-19: dxCompiler resource-retry support
57- 2026-03-17: automatic upgrade on insufficient-resource retry
58- 2026-02-03: retired instance type validation
59- 2026-01-27: dynamic instance type selection

Documentation Index and Releases

The release notes are date-versioned by platform deployment. The skill baseline incorporates:

  • 2026-09-22: dx-toolkit 415.0 Nextflow offline mode
  • 2026-09-15: CLI login default expiration aligned to 18 hours
  • 2026-08-18: normal login sessions shortened to two hours idle, 18 hours absolute
  • 2026-08-11: user API tokens blocked for UKB RAP; dxCompiler 2.18.0 / dxda 0.6.4
  • 2026-08-04: rebuild Nextflow applets to pick up upload-recovery fixes
  • 2026-07-21: dx-toolkit 410.0 malicious-file download warning
  • 2026-07-14: strengthened session/password controls
  • 2026-06-23: dxCompiler 2.17.0 authenticated WDL imports
  • 2026-06-09: file download API securityStatus
  • 2026-05-19: dxCompiler resource-retry support
  • 2026-03-17: automatic upgrade on insufficient-resource retry
  • 2026-02-03: retired instance type validation
  • 2026-01-27: dynamic instance type selection
Raw lines 33 to 60
61## Authentication
63- [Login and Logout](https://documentation.dnanexus.com/user/login-and-logout.md)
64- [Environment Variables](https://documentation.dnanexus.com/user/environment-variables.md)
65- [dx-toolkit environment bootstrap](https://github.com/dnanexus/dx-toolkit/blob/master/environment)
66- [API Authentication](https://documentation.dnanexus.com/developer/api/authentication.md)
67- [API Protocols and Errors](https://documentation.dnanexus.com/developer/api/protocols.md)
69Key current points:
71- Environment variables override saved CLI configuration.
72- `dx env` displays token material.
73- Tokens without explicit expiry default to one month.
74- Token revocation terminates associated active jobs/transfers.
75- New normal-policy login sessions expire after two hours idle and at most
76 18 hours after issuance; org policies may be stricter.
77- User API tokens can be scoped to selected projects and access levels.
78- User-generated API tokens cannot access UKB RAP resources.

Authentication

Key current points:

  • Environment variables override saved CLI configuration.
  • dx env displays token material.
  • Tokens without explicit expiry default to one month.
  • Token revocation terminates associated active jobs/transfers.
  • New normal-policy login sessions expire after two hours idle and at most 18 hours after issuance; org policies may be stricter.
  • User API tokens can be scoped to selected projects and access levels.
  • User-generated API tokens cannot access UKB RAP resources.
Raw lines 61 to 79
80## Apps and Applets
82- [Introduction to Building Apps](https://documentation.dnanexus.com/developer/apps/intro-to-building-apps.md)
83- [Developer Quickstart](https://documentation.dnanexus.com/getting-started/developer-quickstart.md)
84- [App Metadata (`dxapp.json`)](https://documentation.dnanexus.com/developer/apps/app-metadata.md)
85- [I/O and Run Specifications](https://documentation.dnanexus.com/developer/api/running-analyses/io-and-run-specifications.md)
86- [App Execution Environment](https://documentation.dnanexus.com/developer/apps/execution-environment.md)
87- [Apps API](https://documentation.dnanexus.com/developer/api/running-analyses/apps.md)
88- [Applets, Jobs, and Entry Points API](https://documentation.dnanexus.com/developer/api/running-analyses/applets-and-entry-points.md)
89- [App Permissions](https://documentation.dnanexus.com/developer/apps/app-permissions.md)
90- [Types of Errors](https://documentation.dnanexus.com/developer/apps/error-information.md)
91- [Developing Apps and Applets FAQ](https://documentation.dnanexus.com/faqs/developing-apps-and-applets.md)
93Dependencies:
95- [Dependency management](https://documentation.dnanexus.com/developer/apps/dependency-management.md)
96- [Python packages in Ubuntu 24.04 AEE](https://documentation.dnanexus.com/developer/apps/dependency-management/python-package-installation-in-ubuntu-24-04-aee.md)
97- [Asset Build Process](https://documentation.dnanexus.com/developer/apps/dependency-management/asset-build-process.md)
98- [Docker Images](https://documentation.dnanexus.com/developer/apps/dependency-management/using-docker-images.md)
100## Data and Projects
102- [Uploading and Downloading Files](https://documentation.dnanexus.com/user/objects/uploading-and-downloading-files.md)
103- [`dx upload`](https://documentation.dnanexus.com/user/objects/uploading-and-downloading-files/small-sets-of-files/uploading-using-dx.md)
104- [`dx download`](https://documentation.dnanexus.com/user/objects/uploading-and-downloading-files/small-sets-of-files/downloading-using-dx.md)
105- [Upload Agent](https://documentation.dnanexus.com/user/objects/uploading-and-downloading-files/batch/upload-agent.md)
106- [Download Agent](https://documentation.dnanexus.com/user/objects/uploading-and-downloading-files/batch/download-agent.md)
107- [Download Agent repository](https://github.com/dnanexus/dxda)
108- [Download Agent 0.6.4 release](https://github.com/dnanexus/dxda/releases/tag/v0.6.4)
109- [dxFUSE 1.6.1 release](https://github.com/dnanexus/dxfuse/releases/tag/v1.6.1)
110- [Data Object Lifecycle](https://documentation.dnanexus.com/developer/api/data-object-lifecycle.md)
111- [Records API](https://documentation.dnanexus.com/developer/api/introduction-to-data-object-classes/records.md)
112- [Files API](https://documentation.dnanexus.com/developer/api/introduction-to-data-object-classes/files.md)
113- [Data Object Metadata](https://documentation.dnanexus.com/developer/api/introduction-to-data-object-metadata.md)
114- [Folders and Deletion](https://documentation.dnanexus.com/developer/api/data-containers/folders-and-deletion.md)
115- [Cloning](https://documentation.dnanexus.com/developer/api/data-containers/cloning.md)
116- [Project Permissions and Sharing](https://documentation.dnanexus.com/developer/api/data-containers/project-permissions-and-sharing.md)
117- [Project API Methods](https://documentation.dnanexus.com/developer/api/data-containers/projects.md)
118- [Project Data Access Controls](https://documentation.dnanexus.com/getting-started/key-concepts/projects.md#project-data-access-controls)
119- [Search API](https://documentation.dnanexus.com/developer/api/search.md)
120- [Service Limits](https://documentation.dnanexus.com/developer/api/service-limits.md)

Data and Projects

Raw lines 100 to 121
122## Execution and Workflows
124- [Running Apps and Workflows](https://documentation.dnanexus.com/user/running-apps-and-workflows.md)
125- [Running Apps and Applets](https://documentation.dnanexus.com/user/running-apps-and-workflows/running-apps-and-applets.md)
126- [Running Workflows](https://documentation.dnanexus.com/user/running-apps-and-workflows/running-workflows.md)
127- [Monitoring Executions](https://documentation.dnanexus.com/user/running-apps-and-workflows/monitoring-executions.md)
128- [Job Lifecycle](https://documentation.dnanexus.com/user/running-apps-and-workflows/job-lifecycle.md)
129- [Execution Cost and Spending Limits](https://documentation.dnanexus.com/user/running-apps-and-workflows/jobs-and-cost-and-spending-limits.md)
130- [Running Analyses API](https://documentation.dnanexus.com/developer/api/running-analyses.md)
131- [Workflows and Analyses API](https://documentation.dnanexus.com/developer/api/running-analyses/workflows-and-analyses.md)
132- [Building and Running Workflows](https://documentation.dnanexus.com/developer/workflows/building-and-running-workflows.md)
133- [Importing Workflows](https://documentation.dnanexus.com/developer/workflows/importing-workflows.md)
135## WDL, CWL, and Nextflow
137- [dxCompiler repository](https://github.com/dnanexus/dxCompiler)
138- [dxCompiler 2.18.0 release](https://github.com/dnanexus/dxCompiler/releases/tag/2.18.0)
139- [dxCompiler release notes](https://github.com/dnanexus/dxCompiler/blob/develop/RELEASE_NOTES.md)
140- [Running Nextflow Pipelines](https://documentation.dnanexus.com/user/running-apps-and-workflows/running-nextflow-pipelines.md)
142Check these sources together. Platform documentation describes supported
143integration behavior; compiler/tool release notes describe version-specific
144language and runtime changes.

WDL, CWL, and Nextflow

Check these sources together. Platform documentation describes supported integration behavior; compiler/tool release notes describe version-specific language and runtime changes.

Raw lines 135 to 145
146## API Audit Coverage
148- Authentication/protocol: Bearer header, JSON POST body, API version header,
149 project-scoped tokens, session expiry, revocation, and UKB RAP restriction.
150- `/system/findDataObjects`, `/system/findExecutions`, and
151 `/system/findProjects`: request filters, `results`/`next` response shape,
152 cursor continuation, per-page limits, and SDK generator limits.
153- File create/upload/close/describe/download and record create/getDetails/
154 setDetails/close: SDK signatures and official lifecycle/transfer contracts.
155 Object rename/setProperties/addTags and project clone/newFolder/listFolder/
156 move/removeObjects/removeFolder/archive/unarchive were checked against the
157 metadata, container, and project API references. No mutations were executed.
158- `/app/new`, `/applet/new`, app/applet/workflow `run`, `/job/new`, job/analysis
159 `describe` and `terminate`: source manifest versus API payload, accepted
160 classes/access/version values, resource-selector compatibility, return IDs,
161 states, output references, timeout, retry, and cost controls. SDK waits were
162 reviewed in 0.415.0 source; `DXAnalysis.wait_on_done()` can raise while
163 `partially_failed` is still nonterminal.

API Audit Coverage

  • Authentication/protocol: Bearer header, JSON POST body, API version header, project-scoped tokens, session expiry, revocation, and UKB RAP restriction.
  • /system/findDataObjects, /system/findExecutions, and /system/findProjects: request filters, results/next response shape, cursor continuation, per-page limits, and SDK generator limits.
  • File create/upload/close/describe/download and record create/getDetails/ setDetails/close: SDK signatures and official lifecycle/transfer contracts. Object rename/setProperties/addTags and project clone/newFolder/listFolder/ move/removeObjects/removeFolder/archive/unarchive were checked against the metadata, container, and project API references. No mutations were executed.
  • /app/new, /applet/new, app/applet/workflow run, /job/new, job/analysis describe and terminate: source manifest versus API payload, accepted classes/access/version values, resource-selector compatibility, return IDs, states, output references, timeout, retry, and cost controls. SDK waits were reviewed in 0.415.0 source; DXAnalysis.wait_on_done() can raise while partially_failed is still nonterminal.
Raw lines 146 to 164
165## Corrected Legacy Patterns
167The previous skill version contained examples that should not be copied:
169| Legacy pattern | Current guidance |
170|---|---|
171| `license: Unknown` | Skill is MIT; dxpy upstream is Apache-2.0 |
172| Declaring `DX_SECURITY_CONTEXT` in skill metadata | Authenticate through CLI/named secret injection; never expose token |
173| `dx build --app` only | `--app` remains an alias; current help documents `--create-app` |
174| `dxapi` described as required | `dxapi` is optional |
175| `runSpec.systemRequirements` | Deprecated in source manifests; use `regionalOptions.<region>.systemRequirements` |
176| Top-level `resources` | Deprecated; use region-specific `resources` |
177| Static instance list | Discover current regional types; retired types are rejected |
178| `DXFile.open_file()` | Use `dxpy.open_dxfile()` |
179| `name="*.bam"` without mode | Add `name_mode="glob"` |
180| `ResourceNotFound` imported as dxpy exception | Inspect `DXAPIError.name` |
181| Treating every `wait_on_done()` exception as remote failure | In dxpy 0.415.0, `DXJobFailureError` also covers termination and local wait timeout; re-describe state |
182| Runtime `pip install` as primary dependency strategy | Prefer pinned venv/assets/saved images |
183| Pulling floating Docker tags | Pin digest and preferably store `docker save` tarball |
184| `dxpy.dxlink(job.get_output_ref(...))` | Pass `get_output_ref()` directly |

Corrected Legacy Patterns

The previous skill version contained examples that should not be copied:

Legacy patternCurrent guidance
license: UnknownSkill is MIT; dxpy upstream is Apache-2.0
Declaring DX_SECURITY_CONTEXT in skill metadataAuthenticate through CLI/named secret injection; never expose token
dx build --app only--app remains an alias; current help documents --create-app
dxapi described as requireddxapi is optional
runSpec.systemRequirementsDeprecated in source manifests; use regionalOptions.<region>.systemRequirements
Top-level resourcesDeprecated; use region-specific resources
Static instance listDiscover current regional types; retired types are rejected
DXFile.open_file()Use dxpy.open_dxfile()
name="*.bam" without modeAdd name_mode="glob"
ResourceNotFound imported as dxpy exceptionInspect DXAPIError.name
Treating every wait_on_done() exception as remote failureIn dxpy 0.415.0, DXJobFailureError also covers termination and local wait timeout; re-describe state
Runtime pip install as primary dependency strategyPrefer pinned venv/assets/saved images
Pulling floating Docker tagsPin digest and preferably store docker save tarball
dxpy.dxlink(job.get_output_ref(...))Pass get_output_ref() directly
Raw lines 165 to 185
186## Refresh Procedure
188When updating this skill:
1901. Check [PyPI](https://pypi.org/project/dxpy/) for the current dxpy release and
191 Python requirement.
1922. Read platform release notes since the verification date.
1933. Read dx-toolkit and dxCompiler release notes.
1944. Run:
196 ```bash
197 uv run --with "dxpy==<new-version>" \
198 "scripts/inspect_dxpy.py" --strict
199 ```
2015. Compare `dx build --help`, `dx run --help`, and search command help.
2026. Test `dxapp.json` validation and a minimal build in a sandbox project.
2037. Test representative file, job, analysis, WDL/CWL, and Nextflow workflows.
2048. Update the date/version table and skill `metadata.version`.
206Do not update examples from memory alone.

Refresh Procedure

When updating this skill:

  1. Check PyPI for the current dxpy release and Python requirement.
  2. Read platform release notes since the verification date.
  3. Read dx-toolkit and dxCompiler release notes.
  4. Run:

bash uv run --with "dxpy==<new-version>" \ "scripts/inspect_dxpy.py" --strict

  1. Compare dx build --help, dx run --help, and search command help.
  2. Test dxapp.json validation and a minimal build in a sandbox project.
  3. Test representative file, job, analysis, WDL/CWL, and Nextflow workflows.
  4. Update the date/version table and skill metadata.version.

Do not update examples from memory alone.

Raw lines 186 to 206
Runs code scripts/inspect_dxpy.py :198