1.4.0: scale out and connect

Status: proposed · Base: 1.3.0

1.3.0 lets a researcher answer a question with a model. 1.4.0 removes the ceilings on how big that model can be and how close to the real world it can run: distributed populations, GPU and differentiable models, live data and assimilation, multi-scale coupling, and Modelly running in a browser tab. Eleven epics, eight sprints, three pre-releases, all on Windows, macOS and Linux (see Platforms). Backward compatible with 1.3.

Where we start

1.3.0 gives	1.4.0 adds
Experiments with process, thread, Dask, Ray and SLURM executors	One model split across many processes or machines
Calibration (ABC, SMM, Bayesian optimisation)	Gradient-based calibration through differentiable models; history matching
GPU array backend (preview)	GPU backend generally available, with cell spaces on the GPU
Geospatial space and synthetic-population helpers	A full synthetic-population toolkit from census tables
Studio in a local browser	Studio and models running entirely in the browser, shared by a link
Checkpoints and time travel	Live data streams and data assimilation over replicate ensembles

Who 1.4.0 is for

Field	What 1.4.0 unlocks
Epidemiology	Country-scale populations (tens of millions of agents) split over a cluster; nowcasting with live case data through particle filters
Ecology	Landscapes coupled to PDE fields (diffusion of nutrients, heat); GPU runs for large individual-based models
Economics	Agent-based macro models with millions of households; gradient calibration of behavioural parameters
Urban and transport	City digital twins fed by sensor streams; traffic on full regional road networks
Climate and energy	ABMs coupled to system-dynamics or climate emulators; large adoption sweeps on GPUs
Teaching and outreach	A model in a web page, no installation, on any computer with a browser

Epics

Sizes: S under a sprint, M one sprint, L two sprints, XL three or more.

F1 · Distributed simulation (modelly.distributed)

Goal: one model whose population and space are split across processes or machines, with the same results as a single-process run for the same seed.

Platforms: MPI through mpi4py (Open MPI or MPICH on Linux and macOS, MS-MPI on Windows); a Dask and a Ray backend that need no MPI and run on all three.

User stories

• As an epidemiologist, I run a 30-million-agent country model on 64 cores of a cluster with the code I wrote for 100,000 agents on my laptop.

• As an RSE, I check that a distributed run and a single-process run of a small model give the same output, byte for byte.

Design

• Spatial partitioning (grid blocks, graph partitioning for networks, k-d partitioning for continuous space) with ghost cells (halo exchange) at partition borders.

• Agents migrate between partitions when they move; migration happens at a phase boundary, in a deterministic order.

• Random numbers: one stream per agent (from the agent’s seed), so results do not depend on how the space is partitioned.

• Collectors reduce across partitions; checkpoints write one file per partition plus an index.

API sketch

from modelly.distributed import Cluster, partition

with Cluster(backend="mpi", workers=64) as cluster:
    model = cluster.build(Epidemic, scenario, partition=partition.grid_blocks(8, 8))
    model.run_for(365)
    cluster.collect().to_parquet("runs/country.parquet")

Done when: partition-independent results on the zoo’s grid and network models (2, 4 and 16 partitions, identical outputs); weak scaling efficiency of at least 70% from 1 to 64 workers on the grid epidemic playbook; runs on MPI and Dask on all three platforms.

Tasks

• [ ] Per-agent random streams (opt-in in 1.4, default in 2.0)

• [ ] Partitioners: grid blocks, graph (METIS-style, pure Python fallback), k-d

• [ ] Halo exchange and agent migration protocol

• [ ] MPI, Dask and Ray backends

• [ ] Distributed collectors and checkpoints

• [ ] Partition-independence test suite; scaling benchmark

• [ ] Guide: from laptop to cluster

Size: XL · Depends on: 1.3 E3, E11

F2 · GPU backend, generally available

Goal: column phases, cohorts and cell spaces run on a GPU with one setting, with results equal to numpy within documented tolerances.

Platforms: CUDA through CuPy on Linux and Windows; JAX on Linux, Windows (CPU or CUDA) and macOS (Metal where available, CPU otherwise). Every GPU path has a CPU fallback.

Work items: array-API backend for every column operation; CellIndex adjacency, counts and random-neighbour draws on the GPU; CohortTable binomial thinning on the GPU; counter-based random streams (Philox) so GPU and CPU draw the same numbers; memory planner for populations larger than GPU memory (batches); benchmark playbooks with a modelly-gpu variant.

Done when: grid epidemic and insect cohorts playbooks run on CUDA and Metal; results equal the CPU run for integer state and within a stated tolerance for floats; at least 10x faster than the CPU column variant at 10 million agents on a mid-range GPU (measured and published, not assumed).

Size: L · Depends on: 1.3 E11 preview

F3 · Differentiable models (modelly.diff)

Goal: gradients of model outputs with respect to parameters, for gradient-based calibration and sensitivity, where the model allows it.

Platforms: JAX on all three (CPU everywhere, GPU per F2).

Design: column phases traced with JAX; discrete random choices handled with documented estimators (straight-through, Gumbel-softmax, score function) and an explicit warning that the gradient is an approximation; a differentiable flag per phase; gradient checks against finite differences.

User stories: as an economist, I fit 40 behavioural parameters with gradient descent in minutes instead of an ABC run of days; as a modeller, I get local sensitivities (gradients) at the calibrated point for free.

Done when: gradient-based calibration recovers known parameters on two zoo models; gradients agree with finite differences within tolerance; docs explain when gradients are meaningful and when they are not.

Size: L · Depends on: F2, 1.3 E1

F4 · Modelly in the browser (Pyodide and WebAssembly)

Goal: run a Modelly model and Studio entirely in a browser tab, with no installation, and share it as a link or a static page.

Platforms: any OS with a current browser (Chrome, Edge, Firefox, Safari); Pyodide build of the pure wheel in 1.3 (smoke test), compiled modules built for WebAssembly in 1.4.

Work items: Emscripten builds of fastrng, fastloop, fastcell; a modelly export-web command that writes a static site (model code, scenario, Studio) deployable on GitHub Pages; optional dependencies trimmed for the browser; worker threads for long runs; size budget for the first load.

Done when: every 1.3 example runs in the browser from a static export; the export of a zoo model loads in under 10 seconds on a typical connection (measured); the docs gallery embeds live models.

Size: L · Depends on: 1.3 E5 Studio

F5 · Live data and data assimilation (modelly.stream)

Goal: feed real-world observations into running models and keep an ensemble of runs close to reality: the core of a digital twin.

Platforms: pure Python on all three; MQTT and Kafka connectors as optional extras with wheels for all three.

Design: stream sources (files, HTTP polling, MQTT, Kafka); observation operators mapping model state to observed quantities; ensembles of replicates advanced in lock step; assimilation by particle filter (resampling replicates through checkpoints) and ensemble Kalman filter for continuous parameters; a live dashboard in Studio.

User stories: as a public-health modeller, I update tomorrow’s forecast every morning from yesterday’s cases; as a transport researcher, I keep a traffic model in step with loop-detector counts.

Done when: a nowcasting tutorial with a synthetic “truth” run shows the filter tracking it; particle resampling is reproducible for a seed; Studio shows the ensemble band live.

Size: L · Depends on: 1.3 E6 (checkpoint ring), E3

F6 · Multi-scale coupling (modelly.couple)

Goal: couple agents to continuous processes: system-dynamics stocks and flows, ODEs, and PDE fields over space.

Platforms: numpy and scipy on all three.

Design: a Field type (a property layer with a solver: diffusion, advection, decay) stepped on its own time step; ODE and stock-flow components through scipy’s integrators; a coupling schedule (who reads what, when) checked for consistency at build time.

Done when: an ecology model with a diffusing nutrient field and a disease model coupled to a hospital-capacity ODE are in the zoo, with conservation checks (mass balance) as tests.

Size: M · Depends on: 1.3 E8 (rasters)

F7 · Emulators, active learning and history matching

Goal: answer questions about expensive models with far fewer runs.

Design: Gaussian-process and neural emulators trained on experiment results (1.3 E3); active-learning designs that choose the next runs where the emulator is least certain; history matching (implausibility measures, waves of non-implausible space), widely used in epidemiology and climate science.

Done when: history matching reproduces a published wave sequence on a zoo model; active learning reaches the same emulator accuracy as a Latin hypercube with at most half the runs on two benchmarks.

Size: M · Depends on: 1.3 E1, E2, E3

F8 · Synthetic populations (modelly.population)

Goal: realistic starting populations from aggregate data, with households and places.

Design: iterative proportional fitting and combinatorial optimisation from census marginals; households, schools and workplaces assigned with distance constraints; contact networks generated from places; privacy notes and no bundled personal data.

Done when: a synthetic population for a sample region reproduces its marginals within tolerance; the age-structured SEIR zoo model starts from it.

Size: M · Depends on: 1.3 E8, E9

F9 · Performance II

Work items: compiled event scheduler (heap in C, same ordering); compiled DataCollector gather for column-backed agents; memory-mapped columns for populations larger than RAM; per-compiler PGO (GCC, Clang, MSVC) shipped where it measures faster; modelly bench comparison against 1.3 on all three OSs.

Done when: no playbook slower than 1.3 on any OS; event-heavy models (wolf-sheep) measurably faster; a 100-million-row column run on a 16 GB machine.

Size: L

F10 · Studio II

Work items: ensemble views (bands across replicates); experiment browser over the results store; calibration and assimilation dashboards; map and network views at scale (WebGL); recording a session as a reproducible script.

Size: M · Depends on: F4, F5

F11 · Quality and platforms

Work items: partition-independence and GPU-equality suites in the determinism suite; WebAssembly in the release matrix; distributed tests on a three-OS MPI matrix (MS-MPI on Windows); deprecations for 2.0 announced in this release’s notes.

Size: M (continuous)

Priorities

Priority	Epics
Must	F1 Distributed, F2 GPU GA, F5 Live data, F11 Quality
Should	F3 Differentiable, F4 Browser, F7 Emulators, F9 Performance II
Could	F6 Coupling, F8 Synthetic populations, F10 Studio II
Won’t	Breaking changes; a hosted service

        flowchart LR
    E3["1.3 Experiments"] --> F1[F1 Distributed]
    E11["1.3 Performance"] --> F2[F2 GPU]
    F2 --> F3[F3 Differentiable]
    E1["1.3 Calibration"] --> F3
    E1 --> F7[F7 Emulators]
    E5["1.3 Studio"] --> F4[F4 Browser]
    E6["1.3 Time travel"] --> F5[F5 Live data]
    F4 --> F10[F10 Studio II]
    F5 --> F10
    E8["1.3 Geospatial"] --> F6[F6 Coupling]
    E8 --> F8[F8 Populations]
    

Sprint plan

Sprint	Goal	Main items	Exit
S1	Foundations	Per-agent random streams; grid-block partitioner; stream sources and observation operators	Same results with per-agent streams on 1 partition
S2	Distributed core	Halo exchange, migration, MPI backend; distributed collectors	Partition-independent grid epidemic on 2 and 4 partitions
S3	GPU and assimilation · a1	Array-API backend complete; Philox streams; particle filter	GPU equals CPU on integer state; filter tracks synthetic truth
S4	Backends and gradients	Dask and Ray distributed backends; graph partitioner; JAX tracing; ensemble Kalman filter	Distributed runs on Dask on all three OSs; gradient check passes
S5	Browser	Emscripten builds; export-web; Pyodide CI	Every example runs from a static export
S6	Emulators and Studio II · b1	GP emulator, active learning, history matching; ensemble views	Feature complete; API review closed
S7	Coupling and populations · rc1	Fields and ODE coupling; synthetic populations; compiled scheduler	No open blocker
S8	Hardening · 1.4.0	Determinism, partition and GPU suites on all three OSs; scaling report; docs	All release gates pass

Release gates

Release	Gate
1.4.0a1	Distributed grid model partition-independent; GPU backend equals CPU on integer state; wheels for all platforms
1.4.0b1	Must and Should epics feature complete; public API frozen
1.4.0rc1	No open blocker; tutorials run in the docs build
1.4.0	Determinism, partition-independence and GPU suites green on Windows, macOS and Linux; scaling report published

Success metrics

Metric	1.3.0	1.4.0 target
Largest tested population	single machine	30 million agents across 64 workers
Weak scaling efficiency, 1 to 64 workers	not applicable	at least 70%
Platforms with compiled wheels	8	9 (plus WebAssembly)
Calibration methods	ABC, SMM, Bayesian optimisation	plus gradient-based and history matching
Examples runnable in a browser	0	all

Risks

Risk	Likelihood	Impact	Mitigation
Partition independence is hard to guarantee for every space	High	High	Per-agent streams and phase-boundary migration; models that cannot be partitioned are detected and refused with a clear error
MS-MPI on Windows lags Open MPI	Medium	Medium	Dask and Ray backends are first-class; MPI tests on Windows run a reduced matrix
GPU numerics differ from CPU	High	Medium	Integer state compared exactly; floats with stated tolerances; Philox streams shared by CPU and GPU
WebAssembly bundle too large	Medium	Medium	Size budget per release; lazy loading of optional modules
Scope (11 epics)	High	High	Could items move to 1.5 first

Open questions

• [ ] Default distributed backend: MPI (fastest on clusters) or Dask (easiest everywhere)?

• [ ] Per-agent random streams: opt-in in 1.4 and default in 2.0, or default already in 1.4?

• [ ] JAX as an optional extra only, or part of rec for GPU users?

• [ ] Which public data sets to use for the synthetic-population tutorial (licences)?

Sprint review log

Sprint	Shipped	Slipped	Notes
S1
S2
S3
S4
S5
S6
S7
S8