Skill Tracer
Skills › Scientific Agent Skills › fluidsim › installation.md

installation.md

273 lines · skills/fluidsim/references/installation.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Installation, native dependencies, and backends
2

Installation, native dependencies, and backends

Raw lines 1 to 2
3## Supported baseline
4
5Verified 2026-10-01:
6
7- `fluidsim==0.9.0`, released on PyPI 2025-12-04.
8- FluidSim package metadata requires Python `>=3.11` and classifies Python
9 3.11–3.14.
10- `fluidsim-core==0.9.0`, released 2025-12-03.
11- `fluidfft==0.4.5`, released on PyPI 2025-10-14, requires Python `>=3.11`.
12- `pyFFTW==0.15.1`, released 2025-10-22, requires Python `>=3.11`.
13- `mpi4py==4.1.2`, released 2026-05-16, requires Python `>=3.8`.
15The FluidSim installation page still says Python `>=3.9`; current PyPI and
16`pyproject.toml` metadata say `>=3.11`. Use the package metadata for 0.9.0.
18FluidSim 0.9.0 declares:
20- Core: `fluidsim-core>=0.8.6,<0.9.1`, `h5py`, `h5netcdf`,
21 `transonic>=0.6.2`, `xarray`, `rich`, `matplotlib>=3.3`, and `scipy`.
22- `fft`: `pyfftw>=0.10.4`, `fluidfft>=0.4.0`.
23- `mpi`: `mpi4py`.
24- Other extras: `test`, `test-mpi`, and `pulp`.
26The broad upstream constraints are compatibility ranges, not a reproducible
27environment. Record the generated lock and artifact hashes.

Supported baseline

Verified 2026-10-01:

  • fluidsim==0.9.0, released on PyPI 2025-12-04.
  • FluidSim package metadata requires Python >=3.11 and classifies Python 3.11–3.14.
  • fluidsim-core==0.9.0, released 2025-12-03.
  • fluidfft==0.4.5, released on PyPI 2025-10-14, requires Python >=3.11.
  • pyFFTW==0.15.1, released 2025-10-22, requires Python >=3.11.
  • mpi4py==4.1.2, released 2026-05-16, requires Python >=3.8.

The FluidSim installation page still says Python >=3.9; current PyPI and pyproject.toml metadata say >=3.11. Use the package metadata for 0.9.0.

FluidSim 0.9.0 declares:

  • Core: fluidsim-core>=0.8.6,<0.9.1, h5py, h5netcdf, transonic>=0.6.2, xarray, rich, matplotlib>=3.3, and scipy.
  • fft: pyfftw>=0.10.4, fluidfft>=0.4.0.
  • mpi: mpi4py.
  • Other extras: test, test-mpi, and pulp.

The broad upstream constraints are compatibility ranges, not a reproducible environment. Record the generated lock and artifact hashes.

Raw lines 3 to 28
29## Reproducible uv environment
31Preferred project workflow:
33```bash
34uv init --python 3.12
35uv add "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"
36uv lock
37uv sync --frozen
38```
40Check the lock into the study repository. Record:
42- `uv.lock` SHA-256 and target platform.
43- Python implementation/build.
44- FluidSim, FluidSim Core, FluidDyn, FluidFFT, Transonic, Pythran, NumPy,
45 SciPy, h5py, h5netcdf, xarray, and pyFFTW versions.
46- Wheel/sdist hashes and package index.
47- Compiler and native-library versions if any package builds locally.
49For an isolated smoke environment:
51```bash
52uv venv --python 3.12
53uv pip install "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"
54```
56This pins direct dependencies but does not replace a lock for transitive
57reproducibility.
59Bare `fluidsim==0.9.0` supports parts of the framework and analysis stack, but a
60verified local smoke test found that importing NS2D succeeded while
61`Simul.create_default_params()` failed without `fluidfft`. Install the `fft`
62extra for pseudospectral solvers.
64Runtime coverage on 2026-10-01: Python 3.12.10, FluidDyn 0.9.0, NumPy 2.5.3,
65SciPy 1.18.1, h5py 3.16.0, h5netcdf 1.8.1, Transonic 0.8.0, and xarray 2026.9.0.
66Sequential pyFFTW construction, transforms, state serialization, output classes,
67and restart were executed. Native-plugin/MPI installation commands below are
68illustrative; no compiler, distributed allocation, or GPU was exercised.

Reproducible uv environment

Preferred project workflow:

uv init --python 3.12
uv add "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"
uv lock
uv sync --frozen

Check the lock into the study repository. Record:

  • uv.lock SHA-256 and target platform.
  • Python implementation/build.
  • FluidSim, FluidSim Core, FluidDyn, FluidFFT, Transonic, Pythran, NumPy, SciPy, h5py, h5netcdf, xarray, and pyFFTW versions.
  • Wheel/sdist hashes and package index.
  • Compiler and native-library versions if any package builds locally.

For an isolated smoke environment:

uv venv --python 3.12
uv pip install "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"

This pins direct dependencies but does not replace a lock for transitive reproducibility.

Bare fluidsim==0.9.0 supports parts of the framework and analysis stack, but a verified local smoke test found that importing NS2D succeeded while Simul.create_default_params() failed without fluidfft. Install the fft extra for pseudospectral solvers.

Runtime coverage on 2026-10-01: Python 3.12.10, FluidDyn 0.9.0, NumPy 2.5.3, SciPy 1.18.1, h5py 3.16.0, h5netcdf 1.8.1, Transonic 0.8.0, and xarray 2026.9.0. Sequential pyFFTW construction, transforms, state serialization, output classes, and restart were executed. Native-plugin/MPI installation commands below are illustrative; no compiler, distributed allocation, or GPU was exercised.

Raw lines 29 to 69
70## Sequential FFT choices
72The non-compiling path is:
74```bash
75uv add "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"
76```
78FluidFFT 0.4.5 registers:
80- `fft2d.with_pyfftw`
81- `fft3d.with_pyfftw`
82- `fft2d.with_dask` is advertised by the core distribution even when the
83 optional Dask dependency is absent; an advertised method can fail to import.
85The native FFTW plugin is separately versioned:
87```bash
88uv add "fluidfft-fftw==0.0.1"
89```
91The `0.0.1` plugin versions are stable PyPI releases from February 2024 and are
92versioned independently from FluidFFT 0.4.5; they are not proof of compatibility
93with a particular native stack. Before installation, verify that each PyPI
94project links to the official `fluiddyn/fluidfft` monorepo, review the plugin
95source, resolve through `uv.lock`, retain artifact hashes, and use
96`uv sync --frozen`. Do not trust a familiar distribution name alone.
98It provides:
100- `fft2d.with_fftw1d`
101- `fft2d.with_fftw2d`
102- `fft3d.with_fftw3d`
104It requires discoverable FFTW headers/libraries and a working native build
105toolchain. pyFFTW wheels bundle supported binaries on many 64-bit platforms;
106source builds require FFTW `>=3.3`, Cython, and a compiler.
108Discover registered plugin entry points on the current host:
110```bash
111fluidfft-get-methods
112```
114`get_methods()`/`fluidfft-get-methods` list entry points without importing their
115implementations. In the reviewed environment `fft2d.with_dask` was listed but
116`import_fft_class("fft2d.with_dask")` failed because Dask was absent. Do not
117equate discovery with a usable dependency stack or ABI. Run a tiny transform
118and FluidSim pilot, then record the selected method.

Sequential FFT choices

The non-compiling path is:

uv add "fluidsim[fft]==0.9.0" "fluidfft==0.4.5" "pyFFTW==0.15.1"

FluidFFT 0.4.5 registers:

  • fft2d.with_pyfftw
  • fft3d.with_pyfftw
  • fft2d.with_dask is advertised by the core distribution even when the optional Dask dependency is absent; an advertised method can fail to import.

The native FFTW plugin is separately versioned:

uv add "fluidfft-fftw==0.0.1"

The 0.0.1 plugin versions are stable PyPI releases from February 2024 and are versioned independently from FluidFFT 0.4.5; they are not proof of compatibility with a particular native stack. Before installation, verify that each PyPI project links to the official fluiddyn/fluidfft monorepo, review the plugin source, resolve through uv.lock, retain artifact hashes, and use uv sync --frozen. Do not trust a familiar distribution name alone.

It provides:

  • fft2d.with_fftw1d
  • fft2d.with_fftw2d
  • fft3d.with_fftw3d

It requires discoverable FFTW headers/libraries and a working native build toolchain. pyFFTW wheels bundle supported binaries on many 64-bit platforms; source builds require FFTW >=3.3, Cython, and a compiler.

Discover registered plugin entry points on the current host:

fluidfft-get-methods

get_methods()/fluidfft-get-methods list entry points without importing their implementations. In the reviewed environment fft2d.with_dask was listed but import_fft_class("fft2d.with_dask") failed because Dask was absent. Do not equate discovery with a usable dependency stack or ABI. Run a tiny transform and FluidSim pilot, then record the selected method.

Raw lines 70 to 119
120## MPI and distributed FFT
122Nothing in this skill launches MPI or submits a scheduler job. First identify:
124- Site MPI implementation and version: Open MPI, MPICH derivative, Intel MPI,
125 Cray MPICH, or another vendor stack.
126- Compiler wrappers and ABI.
127- FFTW and `fftw3_mpi` versions/build options.
128- Scheduler, process placement, cores/rank, threads/rank, memory/rank, wall time,
129 filesystem, and module/container environment.
131Then lock Python packages:
133```bash
134uv add "mpi4py==4.1.2" \
135 "fluidfft-mpi-with-fftw==0.0.1" \
136 "fluidfft-fftwmpi==0.0.1"
137uv lock
138```
140Plugin methods:
142- `fluidfft-mpi-with-fftw==0.0.1`:
143 `fft2d.mpi_with_fftw1d`, `fft3d.mpi_with_fftw1d`.
144- `fluidfft-fftwmpi==0.0.1`:
145 `fft2d.mpi_with_fftwmpi2d`, `fft3d.mpi_with_fftwmpi3d`.
146- `fluidfft-p3dfft==0.0.1`:
147 `fft3d.mpi_with_p3dfft`, requiring P3DFFT.
148- FluidFFT also declares `pfft` and `p3dfft` extras. Both require separately
149 installed native MPI FFT libraries.
151The package names use hyphens on PyPI; FluidFFT's optional dependency keys map
152to distributions such as `fluidfft-mpi_with_fftw`. Let the lock resolve the
153canonical distribution and preserve it.
155`mpi4py` wheels still need a compatible MPI runtime. Convenience MPI wheels can
156lack GPU awareness or site fabric support; mpi4py recommends system/vendor MPI
157for production. Never mix an `mpi4py` build from one implementation with a
158different launcher/runtime. Verify import and rank identity inside a manually
159allocated tiny job before FluidSim.
161The primary FluidFFT paper shows that the fastest backend depends on array
162shape, machine, and process count; one-dimensional decomposition can be useful
163at low rank count, while pencil/two-dimensional decomposition is needed to
164avoid decomposition limits at high rank count. These are benchmark-context
165claims, not universal backend recommendations.

MPI and distributed FFT

Nothing in this skill launches MPI or submits a scheduler job. First identify:

  • Site MPI implementation and version: Open MPI, MPICH derivative, Intel MPI, Cray MPICH, or another vendor stack.
  • Compiler wrappers and ABI.
  • FFTW and fftw3_mpi versions/build options.
  • Scheduler, process placement, cores/rank, threads/rank, memory/rank, wall time, filesystem, and module/container environment.

Then lock Python packages:

uv add "mpi4py==4.1.2" \
  "fluidfft-mpi-with-fftw==0.0.1" \
  "fluidfft-fftwmpi==0.0.1"
uv lock

Plugin methods:

  • fluidfft-mpi-with-fftw==0.0.1: fft2d.mpi_with_fftw1d, fft3d.mpi_with_fftw1d.
  • fluidfft-fftwmpi==0.0.1: fft2d.mpi_with_fftwmpi2d, fft3d.mpi_with_fftwmpi3d.
  • fluidfft-p3dfft==0.0.1: fft3d.mpi_with_p3dfft, requiring P3DFFT.
  • FluidFFT also declares pfft and p3dfft extras. Both require separately installed native MPI FFT libraries.

The package names use hyphens on PyPI; FluidFFT's optional dependency keys map to distributions such as fluidfft-mpi_with_fftw. Let the lock resolve the canonical distribution and preserve it.

mpi4py wheels still need a compatible MPI runtime. Convenience MPI wheels can lack GPU awareness or site fabric support; mpi4py recommends system/vendor MPI for production. Never mix an mpi4py build from one implementation with a different launcher/runtime. Verify import and rank identity inside a manually allocated tiny job before FluidSim.

The primary FluidFFT paper shows that the fastest backend depends on array shape, machine, and process count; one-dimensional decomposition can be useful at low rank count, while pencil/two-dimensional decomposition is needed to avoid decomposition limits at high rank count. These are benchmark-context claims, not universal backend recommendations.

Raw lines 120 to 166
167## GPU status
169The 2019 FluidFFT paper describes a cuFFT path, and the repository README still
170lists cuFFT. However:
172- FluidFFT 0.4.5 `pyproject.toml` declares no CUDA dependency/extra or cuFFT
173 plugin entry point.
174- The current supported-library page's CUDA section is an unfinished TODO.
175- FluidSim 0.9.0 declares no GPU extra.
177Therefore there is no supported one-line GPU installation in this skill. Do not
178install `nvidia-cufft-*` and claim FluidSim acceleration: a runtime library alone
179does not provide a registered FluidFFT method. A GPU experiment must pin CUDA,
180driver, compiler, plugin source revision, Python packages, precision, hardware,
181and validation tests separately.

GPU status

The 2019 FluidFFT paper describes a cuFFT path, and the repository README still lists cuFFT. However:

  • FluidFFT 0.4.5 pyproject.toml declares no CUDA dependency/extra or cuFFT plugin entry point.
  • The current supported-library page's CUDA section is an unfinished TODO.
  • FluidSim 0.9.0 declares no GPU extra.

Therefore there is no supported one-line GPU installation in this skill. Do not install nvidia-cufft-* and claim FluidSim acceleration: a runtime library alone does not provide a registered FluidFFT method. A GPU experiment must pin CUDA, driver, compiler, plugin source revision, Python packages, precision, hardware, and validation tests separately.

Raw lines 167 to 182
183## Native build prerequisites
185Depending on selected plugins:
187- C/C++11 and sometimes Fortran compilers.
188- Meson/meson-python, Ninja, Pythran, Transonic, Cython, and development headers.
189- FFTW3, threaded FFTW, and/or FFTW MPI.
190- MPI compiler wrappers and runtime.
191- PFFT or P3DFFT headers/libraries.
192- BLAS configuration used by NumPy/Pythran.
193- `CPATH`, `LIBRARY_PATH`, and runtime loader paths where site modules do not
194 provide them.
196The P3DFFT plugin also recognizes `P3DFFT_DIR`, or
197`P3DFFT_LIB_DIR`/`P3DFFT_INCLUDE_DIR`. Record values but never alter global shell
198startup files automatically.

Native build prerequisites

Depending on selected plugins:

  • C/C++11 and sometimes Fortran compilers.
  • Meson/meson-python, Ninja, Pythran, Transonic, Cython, and development headers.
  • FFTW3, threaded FFTW, and/or FFTW MPI.
  • MPI compiler wrappers and runtime.
  • PFFT or P3DFFT headers/libraries.
  • BLAS configuration used by NumPy/Pythran.
  • CPATH, LIBRARY_PATH, and runtime loader paths where site modules do not provide them.

The P3DFFT plugin also recognizes P3DFFT_DIR, or P3DFFT_LIB_DIR/P3DFFT_INCLUDE_DIR. Record values but never alter global shell startup files automatically.

Raw lines 183 to 199
200## HDF5 and netCDF4
202FluidSim 0.9 physical-state files default to netCDF4/HDF5 `.nc`; spectra remain
203HDF5 `.h5`. Standard h5py wheels are usually non-MPI, which is normally
204appropriate because output is coordinated by FluidSim. Parallel HDF5 is a
205separate native build requiring:
207- MPI-enabled HDF5.
208- `h5py` built from source against the same MPI.
209- Matching compiler wrappers and runtime libraries.
211Do not build MPI-enabled h5py merely because the simulation uses MPI. Confirm
212the intended I/O path and test a tiny file first.

HDF5 and netCDF4

FluidSim 0.9 physical-state files default to netCDF4/HDF5 .nc; spectra remain HDF5 .h5. Standard h5py wheels are usually non-MPI, which is normally appropriate because output is coordinated by FluidSim. Parallel HDF5 is a separate native build requiring:

  • MPI-enabled HDF5.
  • h5py built from source against the same MPI.
  • Matching compiler wrappers and runtime libraries.

Do not build MPI-enabled h5py merely because the simulation uses MPI. Confirm the intended I/O path and test a tiny file first.

Raw lines 200 to 213
214## Runtime paths and backend selection
216Official variables:
218```bash
219export FLUIDSIM_PATH="/approved/bounded/results-root"
220export FLUIDDYN_PATH_SCRATCH="/approved/bounded/scratch-root"
221export FLUIDSIM_TYPE_FFT2D="fft2d.with_pyfftw"
222export FLUIDSIM_TYPE_FFT3D="fft3d.with_pyfftw"
223```
225Set only after checking:
227- Paths exist or will be created in an approved parent.
228- No symlink redirects outside the allocation.
229- Quota and inode limits cover the estimate.
230- The method appears in `fluidfft-get-methods`.
231- Scratch retention and purge policy are recorded.
233Prefer `params.oper.type_fft` for an explicit per-run choice. Environment
234variables affect process-wide behavior and must be captured in provenance.
236FluidFFT is also sensitive to `TRANSONIC_BACKEND`; changing it changes generated
237code/performance and belongs in provenance.

Runtime paths and backend selection

Official variables:

export FLUIDSIM_PATH="/approved/bounded/results-root"
export FLUIDDYN_PATH_SCRATCH="/approved/bounded/scratch-root"
export FLUIDSIM_TYPE_FFT2D="fft2d.with_pyfftw"
export FLUIDSIM_TYPE_FFT3D="fft3d.with_pyfftw"

Set only after checking:

  • Paths exist or will be created in an approved parent.
  • No symlink redirects outside the allocation.
  • Quota and inode limits cover the estimate.
  • The method appears in fluidfft-get-methods.
  • Scratch retention and purge policy are recorded.

Prefer params.oper.type_fft for an explicit per-run choice. Environment variables affect process-wide behavior and must be captured in provenance.

FluidFFT is also sensitive to TRANSONIC_BACKEND; changing it changes generated code/performance and belongs in provenance.

Raw lines 214 to 238
239## Verification ladder
241Run in this order:
2431. Dependency-free bundled CLI helps.
2442. Import/version/solver parameter smoke in an isolated pinned environment.
2453. Tiny serial `8x8` or `16x16` no-output initialization and one step.
2464. Tiny serial output round-trip and restart.
2475. Backend-specific FFT test.
2486. Manually allocated two-rank smoke, only if MPI is required.
2497. Representative bounded pilot with resource monitoring.
251Do not run the full upstream test suite or MPI tests on a login node without
252approval; they can compile, spawn processes, and consume resources.

Verification ladder

Run in this order:

  1. Dependency-free bundled CLI helps.
  2. Import/version/solver parameter smoke in an isolated pinned environment.
  3. Tiny serial 8x8 or 16x16 no-output initialization and one step.
  4. Tiny serial output round-trip and restart.
  5. Backend-specific FFT test.
  6. Manually allocated two-rank smoke, only if MPI is required.
  7. Representative bounded pilot with resource monitoring.

Do not run the full upstream test suite or MPI tests on a login node without approval; they can compile, spawn processes, and consume resources.

Raw lines 239 to 253
Starts an agent spawn :252
254## Sources (verified 2026-10-01)
256- [FluidSim PyPI](https://pypi.org/project/fluidsim/) — 0.9.0 metadata and
257 2025-12-04 release.
258- [FluidSim 0.9 source metadata](https://github.com/fluiddyn/fluidsim/blob/0.9.0/pyproject.toml)
259 — dependencies, extras, entry points, Python requirement.
260- [Install and configure](https://fluidsim.readthedocs.io/en/latest/install.html)
261 — extras, native plugins, MPI/HDF5, and environment variables.
262- [FluidFFT 0.4.5 source metadata](https://github.com/fluiddyn/fluidfft/blob/branch/default/pyproject.toml)
263 — plugin extras and methods.
264- [Official FluidFFT plugin source tree](https://github.com/fluiddyn/fluidfft/tree/branch/default/plugins)
265 — provenance for separately distributed native plugins.
266- [FluidFFT plugins](https://fluidfft.readthedocs.io/en/latest/plugins.html) and
267 [installation](https://fluidfft.readthedocs.io/en/latest/install.html).
268- [pyFFTW PyPI](https://pypi.org/project/pyFFTW/) — 0.15.1 metadata and build
269 requirements.
270- [mpi4py PyPI](https://pypi.org/project/mpi4py/) — 4.1.2 metadata and MPI ABI
271 guidance.
272- [FluidFFT primary paper](https://doi.org/10.5334/jors.238), published
273 2019-04-01 — architecture and scoped backend/scaling benchmarks.

Sources (verified 2026-10-01)

Raw lines 254 to 273