Skip to content
VETA VETA Trading Platform
DOCS v1.56.0 · 1b7a079 Grafana Open platform

Performance

Summary

Section titled “Summary”

Latest run: 2026-05-03  ·  Runner: deno-scaling-driver

| Headline metric | Value | | --- | --- | | Maximum sustained order count tested | 500 concurrent | | Sample success rate at peak | 100% (500/500 orders flowed end-to-end) | | End-to-end p99 (submit → fill) at peak | 283 ms | | Latency growth across 5× scaling | None (p99 held flat) | | Journal ingestion p99 (Kafka → DB) | 10 ms |

The pipeline handles a 5× increase in concurrent order volume with no measurable change in per-order latency. Throughput is bounded by the simulator's market-data tick interval (250 ms), not by algo processing or queue contention.

Methodology

Section titled “Methodology”

The load test exercises the full order pipeline:

client → gateway HTTP → orders.new bus → OMS → orders.routed → algo
       → orders.child → EMS → orders.filled → journal (Postgres)

Each scenario submits N orders (LIMIT strategy, ±2% of mid price) round-robined across 4 trader accounts, paced at ~5 orders/sec/user to stay under per-user rate limits. After publication completes, percentiles are read from journal /metrics/latency, which joins event timestamps per orderId to compute stage-by-stage durations.

The runner used for this dataset is a Deno harness that drives gateway POST /load-test then queries journal /metrics/latency. A k6-driven equivalent (k6/baseline-limit.js) is now committed for reproducibility; see k6/README.md for instructions.

Configuration during this run:

  • Risk engine disabled. The concentration-limit rule rejects every user's first order in any symbol (post-trade ratio = 100% when the user has no holdings); seeding test positions is its own session.
  • Tick interval: 250 ms (default).
  • Services co-located on a single dev workstation against a Dockerised Postgres + Redpanda. Production deployments may differ.

Results

Section titled “Results”
End-to-end latency: submit → fill (lower is better)
0 50 100 150 200 250 300 Latency (ms) 100 orders p50: 158 ms 158 100 orders p95: 277 ms 277 100 orders p99: 291 ms 291 100 orders 250 orders p50: 157 ms 157 250 orders p95: 278 ms 278 250 orders p99: 288 ms 288 250 orders 500 orders p50: 155 ms 155 500 orders p95: 274 ms 274 500 orders p99: 283 ms 283 500 orders
  • p50
  • p95
  • p99
Algo wake + slice generation (routed → child)
0 50 100 150 200 250 300 Latency (ms) 100 orders p50: 143 ms 143 100 orders p95: 254 ms 254 100 orders p99: 266 ms 266 100 orders 250 orders p50: 143 ms 143 250 orders p95: 257 ms 257 250 orders p99: 263 ms 263 250 orders 500 orders p50: 142 ms 142 500 orders p95: 254 ms 254 500 orders p99: 263 ms 263 500 orders
  • p50
  • p95
  • p99
Journal ingestion lag (submit → DB write)
0 2 4 6 8 10 12 Latency (ms) 100 orders p50: 2 ms 2 100 orders p95: 6 ms 6 100 orders p99: 11 ms 11 100 orders 250 orders p50: 2 ms 2 250 orders p95: 5 ms 5 250 orders p99: 9 ms 9 250 orders 500 orders p50: 2 ms 2 500 orders p95: 6 ms 6 500 orders p99: 10 ms 10 500 orders
  • p50
  • p95
  • p99

Full table

Section titled “Full table”
ScenarioStagep50p95p99Max
100 ordersroutedToChild143254266269
100 orderschildToFilled15273438
100 orderssubmittedToFilled158277291293
100 orderssubmittedToArrived261113
250 ordersroutedToChild143257263267
250 orderschildToFilled13222526
250 orderssubmittedToFilled157278288289
250 orderssubmittedToArrived25915
500 ordersroutedToChild142254263267
500 orderschildToFilled13212325
500 orderssubmittedToFilled155274283287
500 orderssubmittedToArrived261022

All values in milliseconds. The full machine-readable dataset, including the zero-latency submittedToRouted stage, is at docs/site/src/data/loadtest/2026-05-03-scaling.json, with a CSV companion alongside.

Findings

Section titled “Findings”

  1. Pipeline latency is invariant under scaling. Going from 100 to 500 concurrent orders moved end-to-end p99 from 291 ms to 283 ms, a 3% decrease (within noise). If the algo's per-tick iteration over the active-order set were the bottleneck, p99 would have grown roughly linearly with order count. It did not.

  2. The bottleneck is market-data cadence, not algo overhead. The routedToChild stage (algo wake to child order published) sits at p99 ~263 ms across all scales, almost exactly the simulator's 250 ms tick interval. Orders arriving mid-tick wait for the next tick to be evaluated. The 250 ms ceiling is fundamental at this configuration; no algo-level optimisation can move it.

  3. An earlier architectural hypothesis was tested and falsified. Discussion in prior sessions proposed a price-level index ("predicate-index") to replace the algos' per-tick for loop, on the assumption that iteration cost would dominate at high order counts. The 100 → 500 scaling test disproves that assumption at the scales we care about. We have explicitly not shipped this optimisation.

  4. Storage layer is well-sized. Journal ingestion (Kafka consume → Postgres write) holds p99 ≤ 22 ms even at the 500-order peak. The batched-write pattern (50 ms flush, 20 rows/transaction) leaves substantial headroom.

  5. EMS execution improves with warmup. childToFilled p99 dropped from 34 ms to 23 ms across the three runs, consistent with V8 JIT and connection-pool warmup. Production fill latency should match the warmer numbers, not the cold ones.

Open questions

Section titled “Open questions”

The dataset above is intentionally narrow. Several axes remain untested:

  • WebSocket order submission. Real GUI traffic submits via the gateway WebSocket, not HTTP. Per-iteration overhead and authentication path costs may differ. A k6 v2 scenario will exercise this.
  • OAuth login under load. The current run pre-shares a token. Per-VU OAuth login is a meaningful real-world cost we have not measured.
  • Market-data cadence sensitivity. Halving the tick interval to 125 ms should approximately halve routedToChild p99. We have not validated this; a single-knob test is straightforward and would inform future configuration.
  • Concurrency above 500. The flat-latency observation is empirically true to 500 orders. The break point, where iteration does start to matter, is somewhere above 500 and below the theoretical 50,000+ where microbenchmarks suggest it would. We have not bracketed this.
  • Real risk-engine enabled. Concentration-limit rule rejects fresh users' first orders. Until a position-seeding pass is added, all load runs disable the risk engine. This is a documentation gap, not a system limitation.
  • Multi-strategy mix. Only LIMIT was tested. TWAP, VWAP, POV and the others have different per-tick processing characteristics; their behaviour at scale is presumed similar but unmeasured.

Reproducing this run

Section titled “Reproducing this run”

The load harness is in k6/ and runs as a one-shot Docker container against any environment:

Terminal window
docker compose --profile loadtest run --rm k6

Live percentiles stream to Prometheus and are visualised on the k6 Prometheus dashboard in the local Grafana instance (port 3000 in the dev container; in production the same dashboard is reachable at https://veta.mnetcs.com/grafana/) for the duration of the run. Summary JSON and CSV are written to docs/site/src/data/loadtest/<date>.json and <date>.csv.

See k6/README.md for environment setup, including how to obtain an admin OAuth token.