SmartTest
Benchmark results
<!-- GENERATED by benchmarks/run.mjs — re-run to update. -->
Conditions: linux 7.2.8-9-omarchy · Intel(R) Core(TM) Ultra 7 356H × 16 · 31 GB RAM · Node 26.8.1 · smarttest 0.9.0. Synthetic layered TypeScript repositories (10 modules per layer, 2 imports per module, one test per module, seeded). Test execution time is excluded. Warm runs follow a new edit to the changed module, so the stored graph is patched, not merely reloaded. End-to-end times include process start and output; targets for those rows are the 1.0.0 performance story (warm < 1 s at 10,000 modules, cold < 1 s at 2,000).
| Operation | Target | small (50) | medium (500) | large (2000) | xlarge (10000) |
|---|---|---|---|---|---|
Warm CLI startup (--version) | < 300 ms | 36 ms met | 35 ms met | 42 ms met | 46 ms met |
Warm analysis, end to end (CLI impact after an edit) | < 1.00 s | 138 ms met | 166 ms met | 252 ms met | 734 ms met |
Cold start, end to end (CLI impact, no index or graph) | < 1.00 s | 161 ms met | 436 ms met | 656 ms met | 1.76 s missed |
Simple diff analysis, warm index (in-process analyze after an edit) | < 500 ms | 27 ms met | 99 ms met | 199 ms met | 654 ms missed |
| Warm graph selection (graph build + impact) | < 200 ms | 1 ms met | 5 ms met | 15 ms met | 173 ms met |
| Standard report generation (HTML) | < 2.00 s | 0 ms met | 1 ms met | 5 ms met | 17 ms met |
| Graph update after a body change (watch: patched) | < 10 ms | 0 ms met | 0 ms met | 0 ms met | 0 ms met |
| Graph build from the analyses (no stored graph) | — | 2 ms | 13 ms | 38 ms | 192 ms |
| Stored graph in warm analyses | — | patched | patched | patched | patched |
| Cold index (first run, parse everything) | — | 78 ms | 306 ms | 486 ms | 1.18 s |
| Warm index (nothing changed) | — | 5 ms | 24 ms | 44 ms | 212 ms |
| Tests selected for a mid-graph change | — | 7 / 50 (14%) | 205 / 500 (41%) | 975 / 2000 (49%) | 4974 / 10000 (50%) |
| Process RSS after run | — | 139 MB | 652 MB | 909 MB | 1626 MB |
Targets are aspirational engineering targets from PRD §31.3 and the 1.0.0 performance story, not guarantees. The cold-start target is stated for 2,000 modules; it is shown for every size. Misses are reported, not hidden.