Skill Tracer
Skills › Scientific Agent Skills › fluidsim › advanced_features.md

advanced_features.md

350 lines · skills/fluidsim/references/advanced_features.md at 5f2bc03

Open on GitHub
1# Forcing, operators, MPI, extensions, and migrations
2

Forcing, operators, MPI, extensions, and migrations

Raw lines 1 to 2
3## Forcing architecture
4
5FluidSim assembles forcing classes through the selected solver's registry.
6NS2D 0.9 advertises:
7
8- `in_script`
9- `in_script_coarse`
10- `pseudo_spectral`
11- `proportional`
12- `tcrandom`
13- `tcrandom_anisotropic`
15Availability and default forced variable are solver-specific.
17Base fields:
19```python
20params.forcing.enable = True
21params.forcing.type = "tcrandom"
22params.forcing.forcing_rate = 1.0
23params.forcing.key_forced = None
24params.forcing.nkmin_forcing = 4
25params.forcing.nkmax_forcing = 5
26params.forcing.tcrandom.time_correlation = "based_on_forcing_rate"
27```
29Current source converts `nkmin_forcing`/`nkmax_forcing` to dimensional
30wave-number bounds using the operator's wave-number spacing. Inspect the
31resulting forced region; the integers are not necessarily physical wave
32numbers.

Forcing architecture

FluidSim assembles forcing classes through the selected solver's registry. NS2D 0.9 advertises:

  • in_script
  • in_script_coarse
  • pseudo_spectral
  • proportional
  • tcrandom
  • tcrandom_anisotropic

Availability and default forced variable are solver-specific.

Base fields:

params.forcing.enable = True
params.forcing.type = "tcrandom"
params.forcing.forcing_rate = 1.0
params.forcing.key_forced = None
params.forcing.nkmin_forcing = 4
params.forcing.nkmax_forcing = 5
params.forcing.tcrandom.time_correlation = "based_on_forcing_rate"

Current source converts nkmin_forcing/nkmax_forcing to dimensional wave-number bounds using the operator's wave-number spacing. Inspect the resulting forced region; the integers are not necessarily physical wave numbers.

Raw lines 3 to 33
34### Normalization
36Current normalized-forcing fields include:
38```python
39params.forcing.normalized.constant_rate_of = None
40params.forcing.normalized.type = "2nd_degree_eq"
41params.forcing.normalized.which_root = "minabs"
42```
44The implementation can solve a quadratic normalization so the time-step-mean
45injection of a quadratic quantity matches `forcing_rate`. The exact quadratic
46quantity and key depend on the solver/forced field. Therefore:
48- State the intended injected invariant and units.
49- Confirm `key_forced`.
50- Measure forcing power in output.
51- Check the global/spectral budget using the same convention.
52- Test time-step sensitivity of the measured injection.
54Do not call `forcing_rate` “energy input” without verifying the selected class.

Normalization

Current normalized-forcing fields include:

params.forcing.normalized.constant_rate_of = None
params.forcing.normalized.type = "2nd_degree_eq"
params.forcing.normalized.which_root = "minabs"

The implementation can solve a quadratic normalization so the time-step-mean injection of a quadratic quantity matches forcing_rate. The exact quadratic quantity and key depend on the solver/forced field. Therefore:

  • State the intended injected invariant and units.
  • Confirm key_forced.
  • Measure forcing power in output.
  • Check the global/spectral budget using the same convention.
  • Test time-step sensitivity of the measured injection.

Do not call forcing_rate “energy input” without verifying the selected class.

Raw lines 34 to 55
56### Time-correlated random forcing
58The 0.9 field is:
60```python
61params.forcing.tcrandom.time_correlation = "based_on_forcing_rate"
62```
64or a positive finite time value. The rate-derived setting requires positive
65`forcing_rate`. Current source derives the default period as a power of
66the forcing rate and stores two random seeds plus the last-change time in state
67parameters under `/state_params/forcing` (`seed0`, `seed1`, `t_last_change`).
68FluidSim 0.9.0 writes these state parameters into restart files;
690.8.6 fixed a time-correlated forcing restart bug.
71For reproducibility, preserve:
73- Initial random seed strategy.
74- Saved forcing state parameters.
75- MPI rank count/decomposition and package versions.
76- Correlation-time setting and measured autocorrelation.
77- Restart boundary diagnostics.

Time-correlated random forcing

The 0.9 field is:

params.forcing.tcrandom.time_correlation = "based_on_forcing_rate"

or a positive finite time value. The rate-derived setting requires positive forcing_rate. Current source derives the default period as a power of the forcing rate and stores two random seeds plus the last-change time in state parameters under /state_params/forcing (seed0, seed1, t_last_change). FluidSim 0.9.0 writes these state parameters into restart files; 0.8.6 fixed a time-correlated forcing restart bug.

For reproducibility, preserve:

  • Initial random seed strategy.
  • Saved forcing state parameters.
  • MPI rank count/decomposition and package versions.
  • Correlation-time setting and measured autocorrelation.
  • Restart boundary diagnostics.
Raw lines 56 to 78
79### In-script forcing
81Use the solver's registered `InScriptForcing*` interface and documented
82`compute_forcing_fft_each_time` or coarse equivalent. Do not monkey-patch a
83method with a lambda copied from an old example:
85- State keys have changed in some solvers.
86- Local spectral layout depends on FFT/MPI backend.
87- Hermitian/reality constraints and normalization must be preserved.
88- A literal global Fourier index is not portable across decompositions.
90Implement a reviewed subclass/extension with unit tests on tiny sequential and
91MPI layouts. Validate zero-net/target injection, symmetry, and budget effects.

In-script forcing

Use the solver's registered InScriptForcing* interface and documented compute_forcing_fft_each_time or coarse equivalent. Do not monkey-patch a method with a lambda copied from an old example:

  • State keys have changed in some solvers.
  • Local spectral layout depends on FFT/MPI backend.
  • Hermitian/reality constraints and normalization must be preserved.
  • A literal global Fourier index is not portable across decompositions.

Implement a reviewed subclass/extension with unit tests on tiny sequential and MPI layouts. Validate zero-net/target injection, symmetry, and budget effects.

Raw lines 79 to 92
93## Operators and array ownership
95`sim.oper` provides solver-selected grids, FFT/IFFT, differentiation, vector
96calculus, projections, spectra, dealiasing, and distributed-array helpers.
97Method names and array layouts depend on operator class.
99Never assume:
101- Axis order from `nx`, `ny`, `nz`.
102- Full global arrays on every rank.
103- A Fourier mode has the same local index under another backend/rank count.
104- All FFT backends use the same spectral shape.
105- A gathered array fits rank-0 memory.
106- Direct NumPy sums have the same normalization as
107 `oper.sum_wavenumbers`.
109Use documented operator methods and inspect:
111```python
112print(type(sim.oper))
113print(sim.oper.axes)
114print(sim.params.oper)
115```
117For custom diagnostics, test sequential and distributed shapes and compare
118against analytical transforms at tiny resolution.

Operators and array ownership

sim.oper provides solver-selected grids, FFT/IFFT, differentiation, vector calculus, projections, spectra, dealiasing, and distributed-array helpers. Method names and array layouts depend on operator class.

Never assume:

  • Axis order from nx, ny, nz.
  • Full global arrays on every rank.
  • A Fourier mode has the same local index under another backend/rank count.
  • All FFT backends use the same spectral shape.
  • A gathered array fits rank-0 memory.
  • Direct NumPy sums have the same normalization as oper.sum_wavenumbers.

Use documented operator methods and inspect:

print(type(sim.oper))
print(sim.oper.axes)
print(sim.params.oper)

For custom diagnostics, test sequential and distributed shapes and compare against analytical transforms at tiny resolution.

Raw lines 93 to 119
120## Dealiasing
122The common Cartesian field is:
124```python
125params.oper.coef_dealiasing = 2 / 3
126params.oper.truncation_shape = "cubic"
127```
129FluidSim also implements phase-shift time schemes. A coefficient or scheme name
130does not prove alias removal for a custom nonlinearity. Verify:
132- Polynomial/nonlinear form and expected alias interactions.
133- Where dealiasing is applied.
134- Truncation geometry.
135- Spectral tails and invariant transfer.
136- Results under stricter truncation or exact phase-shift method.
138Do not combine an aggressive cutoff and high-order dissipation merely to obtain
139a visually smooth spectrum.

Dealiasing

The common Cartesian field is:

params.oper.coef_dealiasing = 2 / 3
params.oper.truncation_shape = "cubic"

FluidSim also implements phase-shift time schemes. A coefficient or scheme name does not prove alias removal for a custom nonlinearity. Verify:

  • Polynomial/nonlinear form and expected alias interactions.
  • Where dealiasing is applied.
  • Truncation geometry.
  • Spectral tails and invariant transfer.
  • Results under stricter truncation or exact phase-shift method.

Do not combine an aggressive cutoff and high-order dissipation merely to obtain a visually smooth spectrum.

Raw lines 120 to 140
141## Time schemes
143Documented pseudospectral names:
145- `Euler`
146- `Euler_phaseshift`
147- `Euler_phaseshift_random`
148- `RK2`
149- `RK2_trapezoid`
150- `RK2_phaseshift`
151- `RK2_phaseshift_random`
152- `RK2_phaseshift_random_split`
153- `RK2_phaseshift_exact`
154- `RK4`
156The implementation treats linear terms with exact coefficients in its
157pseudospectral stepper and evaluates nonlinear tendencies according to the
158named scheme. Verify source and solver coupling before making an order/stability
159claim.
161Always check:
163- Advective CFL.
164- Wave frequency limits (stratification, rotation, shallow-water waves).
165- Diffusive/hyperdiffusive limits.
166- Forcing correlation and output cadence relative to `deltat`.
167- Smaller `cfl_coef`/`deltat_max` comparison.
169`USE_CFL=True` only activates the solver's CFL logic; it does not guarantee all
170accuracy/stability constraints are resolved.

Time schemes

Documented pseudospectral names:

  • Euler
  • Euler_phaseshift
  • Euler_phaseshift_random
  • RK2
  • RK2_trapezoid
  • RK2_phaseshift
  • RK2_phaseshift_random
  • RK2_phaseshift_random_split
  • RK2_phaseshift_exact
  • RK4

The implementation treats linear terms with exact coefficients in its pseudospectral stepper and evaluates nonlinear tendencies according to the named scheme. Verify source and solver coupling before making an order/stability claim.

Always check:

  • Advective CFL.
  • Wave frequency limits (stratification, rotation, shallow-water waves).
  • Diffusive/hyperdiffusive limits.
  • Forcing correlation and output cadence relative to deltat.
  • Smaller cfl_coef/deltat_max comparison.

USE_CFL=True only activates the solver's CFL logic; it does not guarantee all accuracy/stability constraints are resolved.

Raw lines 141 to 171
172## Custom initial conditions
174`in_script` gives direct control, but use current state keys:
1761. Construct `Simul` with `init_fields.type = "in_script"`.
1772. Inspect the solver's state documentation/keys.
1783. Fill canonical physical or spectral variables.
1794. Call the documented conversion in the correct direction.
1805. Apply projection/dealiasing/constraints as required.
1816. Save an initialization checkpoint and verify budgets before stepping.
183Old examples that fill `vx`/`vy` and then call a
184spectral-to-physical conversion can overwrite the intended state. NS2D 0.9
185documentation shows physical keys including `ux`, `uy`, and `rot`; use the
186selected solver's actual keys.

Custom initial conditions

in_script gives direct control, but use current state keys:

  1. Construct Simul with init_fields.type = "in_script".
  2. Inspect the solver's state documentation/keys.
  3. Fill canonical physical or spectral variables.
  4. Call the documented conversion in the correct direction.
  5. Apply projection/dealiasing/constraints as required.
  6. Save an initialization checkpoint and verify budgets before stepping.

Old examples that fill vx/vy and then call a spectral-to-physical conversion can overwrite the intended state. NS2D 0.9 documentation shows physical keys including ux, uy, and rot; use the selected solver's actual keys.

Raw lines 172 to 187
188## Extending a solver
190FluidSim's `InfoSolver`/class registry supports extensions. For a research
191extension:
193- Pin FluidSim/FluidSim Core source and version.
194- Subclass the closest solver and extend default parameters through the current
195 class mechanism.
196- Register state variables, operators, initialization, forcing, outputs, and
197 restart state explicitly.
198- Define nonlinear tendencies with documented sign and normalization.
199- Add unit/manufactured-solution tests and budget identities.
200- Test serialization/restart and old/new parameter merging.
201- Benchmark only after correctness tests.
203Avoid private-method snippets from old versions without source review.

Extending a solver

FluidSim's InfoSolver/class registry supports extensions. For a research extension:

  • Pin FluidSim/FluidSim Core source and version.
  • Subclass the closest solver and extend default parameters through the current class mechanism.
  • Register state variables, operators, initialization, forcing, outputs, and restart state explicitly.
  • Define nonlinear tendencies with documented sign and normalization.
  • Add unit/manufactured-solution tests and budget identities.
  • Test serialization/restart and old/new parameter merging.
  • Benchmark only after correctness tests.

Avoid private-method snippets from old versions without source review.

Raw lines 188 to 204
205## FluidFFT backend selection
207Registered methods are entry points, not proof their dependencies load.
208Discover them:
210```python
211from fluidfft import get_methods
213print(sorted(get_methods(ndim=2)))
214print(sorted(get_methods(ndim=3)))
215```
217Set per run:
219```python
220params.oper.type_fft = "fft2d.with_pyfftw"
221```
223or use `FLUIDSIM_TYPE_FFT2D`/`FLUIDSIM_TYPE_FFT3D` before process start.
224Record actual method and plugin distribution.
226FluidFFT's 2019 primary paper demonstrates:
228- Unified C++/Python APIs for multiple FFT libraries.
229- One-dimensional and pencil/two-dimensional MPI decompositions.
230- Hardware/shape/process-count-dependent fastest methods.
231- Scaling beyond the limits of slab decomposition in the tested cases.
233Do not transfer its fastest-method or wall-time numbers to current hardware.
234Benchmark a bounded representative shape in the target environment.

FluidFFT backend selection

Registered methods are entry points, not proof their dependencies load. Discover them:

from fluidfft import get_methods

print(sorted(get_methods(ndim=2)))
print(sorted(get_methods(ndim=3)))

Set per run:

params.oper.type_fft = "fft2d.with_pyfftw"

or use FLUIDSIM_TYPE_FFT2D/FLUIDSIM_TYPE_FFT3D before process start. Record actual method and plugin distribution.

FluidFFT's 2019 primary paper demonstrates:

  • Unified C++/Python APIs for multiple FFT libraries.
  • One-dimensional and pencil/two-dimensional MPI decompositions.
  • Hardware/shape/process-count-dependent fastest methods.
  • Scaling beyond the limits of slab decomposition in the tested cases.

Do not transfer its fastest-method or wall-time numbers to current hardware. Benchmark a bounded representative shape in the target environment.

Raw lines 205 to 235
236## MPI planning and safety
238Never call `mpirun`, `mpiexec`, `srun`, `qsub`, `sbatch`, OAR tools, or a
239FluidDyn cluster submitter automatically.
241Required preflight:
243- Written resource estimate and output estimate.
244- Approved allocation and partition/account.
245- Exact MPI implementation/ABI and launcher.
246- FFT plugin/native library compatibility.
247- Rank/thread placement and oversubscription check.
248- Per-rank local shapes and no zero-sized unsupported decomposition.
249- Memory/rank and rank-0 gather/output risk.
250- Wall-time signal/checkpoint behavior.
251- Filesystem quota, inode count, stripe policy, and cleanup.
252- Tiny serial then two-rank smoke.
253- Restart plan with immutable parent state.
255Output behavior can differ with MPI-enabled h5py. Standard h5py usually causes
256rank 0 to write assembled state; MPI h5py can use an `mpio` driver. Verify the
257locked h5py build and output path with a tiny test.

MPI planning and safety

Never call mpirun, mpiexec, srun, qsub, sbatch, OAR tools, or a FluidDyn cluster submitter automatically.

Required preflight:

  • Written resource estimate and output estimate.
  • Approved allocation and partition/account.
  • Exact MPI implementation/ABI and launcher.
  • FFT plugin/native library compatibility.
  • Rank/thread placement and oversubscription check.
  • Per-rank local shapes and no zero-sized unsupported decomposition.
  • Memory/rank and rank-0 gather/output risk.
  • Wall-time signal/checkpoint behavior.
  • Filesystem quota, inode count, stripe policy, and cleanup.
  • Tiny serial then two-rank smoke.
  • Restart plan with immutable parent state.

Output behavior can differ with MPI-enabled h5py. Standard h5py usually causes rank 0 to write assembled state; MPI h5py can use an mpio driver. Verify the locked h5py build and output path with a tiny test.

Raw lines 236 to 258
259## Parametric studies
261Do not loop over simulations and start them directly in one script by default.
262Instead:
2641. Materialize one strict config per case.
2652. Assign a stable case ID and seed.
2663. Validate/estimate each case.
2674. Sum aggregate CPU, memory concurrency, disk, files, and wall time.
2685. Generate scripts only.
2696. Review sampling design and avoid changing multiple factors ambiguously.
2707. Submit through an approved external workflow.
2718. Track failures/missing cases without silently resampling.
273Analyze observables with refinement and stochastic uncertainty, not only final
274values.

Parametric studies

Do not loop over simulations and start them directly in one script by default. Instead:

  1. Materialize one strict config per case.
  2. Assign a stable case ID and seed.
  3. Validate/estimate each case.
  4. Sum aggregate CPU, memory concurrency, disk, files, and wall time.
  5. Generate scripts only.
  6. Review sampling design and avoid changing multiple factors ambiguously.
  7. Submit through an approved external workflow.
  8. Track failures/missing cases without silently resampling.

Analyze observables with refinement and stochastic uncertainty, not only final values.

Raw lines 259 to 275
276## Checkpoint and restart
278Physical-state saving is checkpoint creation:
280```python
281params.output.periods_save.phys_fields = 1.0
282```
284But checkpoint usability requires:
286- Complete `/state_phys` datasets.
287- `/info_simul/params` and solver metadata.
288- 0.9 state parameters where needed.
289- Matching solver/grid/domain/state.
290- SHA-256 and parent lineage.
291- Enough disk for parent and child.
293Use the bundled compatibility checker before every continuation. A mechanically
294compatible state can still be scientifically invalid after changed viscosity,
295forcing, timestep, backend, or resolution.

Checkpoint and restart

Physical-state saving is checkpoint creation:

params.output.periods_save.phys_fields = 1.0

But checkpoint usability requires:

  • Complete /state_phys datasets.
  • /info_simul/params and solver metadata.
  • 0.9 state parameters where needed.
  • Matching solver/grid/domain/state.
  • SHA-256 and parent lineage.
  • Enough disk for parent and child.

Use the bundled compatibility checker before every continuation. A mechanically compatible state can still be scientifically invalid after changed viscosity, forcing, timestep, backend, or resolution.

Raw lines 276 to 296
297## Migration notes to 0.9.0

Migration notes to 0.9.0

Raw lines 297 to 298
299### 0.9.0 (release notes dated 2025-12-03)
301- Restart files store state parameters.
302- Added basic physical-field utilities.
303- Fixed restart filenames.
304- Improved post-initialization information and profile analysis.

0.9.0 (release notes dated 2025-12-03)

  • Restart files store state parameters.
  • Added basic physical-field utilities.
  • Fixed restart filenames.
  • Improved post-initialization information and profile analysis.
Raw lines 299 to 305
306### 0.8.6 (2025-11-23)
308- h5netcdf 1.7 compatibility.
309- Fixed incorrect restart for time-correlated forcing.

0.8.6 (2025-11-23)

  • h5netcdf 1.7 compatibility.
  • Fixed incorrect restart for time-correlated forcing.
Raw lines 306 to 310
311### 0.8.5 (2025-10-23)
313- Python 3.14 support.

0.8.5 (2025-10-23)

  • Python 3.14 support.
Raw lines 311 to 314
315### 0.8.2 (2024-08-17)
317- Python 3.12, NumPy 2.0, and mpi4py 4.0 compatibility.

0.8.2 (2024-08-17)

  • Python 3.12, NumPy 2.0, and mpi4py 4.0 compatibility.
Raw lines 315 to 318
319### 0.8.0 (2024-01-31)
321- Meson/meson-python build system.
323Practical migrations from the previous skill:
325- Python baseline: `>=3.11`, not `>=3.9` for 0.9.0.
326- Use exact `fluidsim==0.9.0` and lock dependencies.
327- Pseudospectral defaults need the `fft` extra.
328- Use `time_stepping.cfl_coef`, not `CFL`.
329- Use `forcing.tcrandom.time_correlation`, not a flat field.
330- Use `plate2d`, not `fvk`.
331- Physical states default to `.nc`, not `.h5`; spectra remain `.h5`.
332- Use `spect_energy_budg.h5`, not a timestamped budget glob.
333- Prefer `load_for_restart`/`fluidsim-restart --only-check`; preserve state
334 parameters and hashes.
335- Do not advertise ParaView direct compatibility without an explicit tested
336 conversion/plugin.

0.8.0 (2024-01-31)

  • Meson/meson-python build system.

Practical migrations from the previous skill:

  • Python baseline: >=3.11, not >=3.9 for 0.9.0.
  • Use exact fluidsim==0.9.0 and lock dependencies.
  • Pseudospectral defaults need the fft extra.
  • Use time_stepping.cfl_coef, not CFL.
  • Use forcing.tcrandom.time_correlation, not a flat field.
  • Use plate2d, not fvk.
  • Physical states default to .nc, not .h5; spectra remain .h5.
  • Use spect_energy_budg.h5, not a timestamped budget glob.
  • Prefer load_for_restart/fluidsim-restart --only-check; preserve state parameters and hashes.
  • Do not advertise ParaView direct compatibility without an explicit tested conversion/plugin.
Raw lines 319 to 337
338## Sources (verified 2026-10-01)
340- [FluidSim forcing base source](https://github.com/fluiddyn/fluidsim/blob/0.9.0/fluidsim/base/forcing/base.py).
341- [Specific forcing source](https://github.com/fluiddyn/fluidsim/blob/0.9.0/fluidsim/base/forcing/specific.py).
342- [Pseudospectral time-step API](https://fluidsim.readthedocs.io/en/latest/generated/fluidsim.base.time_stepping.pseudo_spect.html).
343- [FluidSim development tutorial](https://fluidsim.readthedocs.io/en/latest/ipynb/tuto_dev.html).
344- [FluidSim release notes](https://fluidsim.readthedocs.io/en/latest/changes.html).
345- [FluidFFT plugins](https://fluidfft.readthedocs.io/en/latest/plugins.html).
346- [FluidFFT supported libraries](https://fluidfft.readthedocs.io/en/latest/install/fft_libs.html).
347- Mohanan et al., [FluidFFT primary paper](https://doi.org/10.5334/jors.238),
348 published 2019-04-01.
349- Mohanan et al., [FluidSim primary paper](https://doi.org/10.5334/jors.239),
350 published 2019-04-26.