review.md
70 lines · skills/esm/references/review.md at 5f2bc03
Review evidence — 2026-09-30
Raw lines 1 to 2Release and primary sources
The target is the published esm==3.4.1.post1 wheel (uploaded 2026-09-16), not the earlier 3.2.3 API or an unpinned GitHub checkout. The wheel's SHA-256 was checked against official PyPI metadata. Current upstream main documentation was cross-checked against that released source before adopting its examples.
| Primary source | What was verified |
|---|---|
| PyPI release and release JSON | Python >=3.12; Torch >=2.11,<2.12; Transformers >=4.57.6,<5; Linux x86_64 GPU dependency markers |
| Biohub repository | Current local ESMC/ESMFold2 interfaces, Biohub migration, open ESMC6B weights |
| ESM3 README | Local/hosted model IDs, masked generation, clearing tracks before a round trip |
| SDK factories | Distinct ESM3/ESMC/folding factories, Biohub defaults, import-time token capture |
| SDK types | Generation/Folding/Logits configs, per-track output containers, PDB methods |
| Hosted clients | Exact endpoint names, request and response fields, ESMProteinError values, MSA warning, omitted pair-chain IPTM flag |
| HTTP transport and retry | Bearer auth, /api/v1/, httpx, timeout semantics, bounded retries and response decoding |
| Native ESMC, tokenizer, compatibility | Batched token API, special tokens, hidden states, deprecated wrapper differences |
| Generation implementation, function tokenizer, annotation type | Track masking, temperature annealing, supported-label validation, 1-based inclusive annotations |
| Complex input types | Typed polymer/ligand inputs, list-valued CCD IDs, zero-based modification indices |
| ESMC300M, ESMC600M, ESMC6B, ESMFold2 | Model cards, 2048-token ESMC context, weight availability and license notices |
| Legacy Meta ESM | Separate fair-esm distribution and ESM2/ESMFold APIs |
| scikit-learn pitfalls, PCA, KMeans | Leakage boundaries, PCA dimension bounds, explicit clustering settings |
Public Hugging Face model metadata and the three small ESMC config.json files were fetched without authentication or weights. Configs reported dimensions 960/1152/2560, layers 30/36/80, and 2048 maximum position metadata. All parsed with the released EsmcConfig. Public access metadata does not test weight loading, hosted account access or model quality. ESMC cards list MIT plus other with a third-party-notice link; retain those notices rather than treating all dependency licenses as MIT.
Executed checks
The suite in tests/esm/ exercises a two-layer, 32-dimensional random ESMC model on CPU. It checks native batch/individual equivalence, token axes and hidden states, residue-only pooling including NaN padding, malformed inputs and context length, legacy SDK output containers, tiny saved-checkpoint reload, and a synthetic two-residue PDB round trip. Random weights establish software behavior, not learned scientific performance.
httpx.MockTransport exercises actual released ESM3 generation, ESMC encode/logits/decode, and folding transport paths with synthetic responses. It checks host/path, bearer-header construction, request timeout, payload fields, response envelopes, and returned 400/401 error values. Folding serialization checks reproduce the unsupported distogram flag and absent pair-chain IPTM request field. These checks do not call Biohub or use real keys.
The isolated suite uses Python 3.12, esm 3.4.1.post1, Torch 2.11.0 and Transformers 4.57.6. The documentation's tiny native ESMC example, small-dataset clustering recipe (2 and 12 synthetic rows), and variant bookkeeping with a fake generator were also run. Every Python code fence was syntax-checked. The PDB fixture warns about absent entity metadata, expected for its minimal synthetic input; sequence and coordinates round-trip correctly.
Raw lines 34 to 56Verification boundaries
No pretrained model weights, full protein datasets or CUDA toolchains were downloaded. Pretrained ESM3/ESMC/ESMFold2 inference, fine tuning, function-vocabulary assets, Flash Attention, multi-GPU/Fold-CP, authenticated service availability, rate limits, pricing, and scientific performance were not executed. Examples requiring these are explicitly illustrative. The skill does not claim tested SageMaker deployment, direct submitted batch-job workflows, or an API-compatible migration from fair-esm.
The old workflow diagram connected ESMC/t-SNE output to ESMFold2 and labeled folding as esm.sdk.client(). Those relationships are incorrect: ESMC features, ESM3 generation and ESMFold2 folding are distinct routes. A replacement must show their separate input/output contracts and prediction/validation boundary.