Concurrency & capacity sizing

Concurrency is controlled at two layers, and both must be tuned together:

1. Product layer – how many runs and pipelines your users are allowed to run at once. These are customer-facing settings you can change from the UI.
2. Infrastructure layer – how much concurrent work your deployment’s Postgres, Redis, and sandbox quota can actually support. These are operator settings configured on the server.

If the product layer allows more concurrency than the infrastructure layer can support, runs start failing – either gracefully at the product layer (capacity deferral or lock_timeout) or ungracefully at the infrastructure layer (connection errors). This page explains both layers and the sizing relationship between them.

Product layer: controlling concurrent runs

The product layer exposes two customer-facing knobs:

• Pipeline.max_concurrent_runs – the maximum number of runs a single pipeline can execute concurrently. See Capacity & concurrency for how the executor enforces this.
• Org-level sandbox concurrency cap – the maximum number of sandboxes your organisation may hold open at once across all pipelines and runs.

The executor serialises capacity checks via SELECT FOR UPDATE on the pipeline row to prevent race conditions. When a run starts but cannot acquire a capacity slot:

• If the slot becomes free within lock_wait_timeout_seconds, the run is deferred and proceeds when capacity allows.
• If no slot frees up within the timeout, the run fails with error_code="lock_timeout".

Set these product knobs to match the concurrency your users actually need – then verify your infrastructure can support that target (next section). The product knobs are the customer’s contract with your deployment; the infra knobs are the deployment’s contract with Postgres, Redis, and the sandbox provider.

Infrastructure layer: operator knobs

These knobs are deployment configuration, set by the operator on the server. They are intentionally not exposed in the UI – they are not product features, and their correct values depend entirely on the capacity of the Postgres and Redis instances underneath. Tune them to your deployment, not to any published default.

Infrastructure layer: operator knobs

Knob	What it controls	Maps to	How to size
SAQ_WORKER_CONCURRENCY	Number of jobs each worker process executes concurrently	Postgres/Redis load per worker	Multiply by your worker count and verify the result stays under your DB and Redis budgets (below)
SAQ_REDIS_POOL_SIZE	Max Redis connections held open per worker process (connection pool)	Redis maxclients / connection budget	workers × pool_size must stay well under Redis maxclients, leaving headroom for the web layer
SAQ_WORKER_DB_POOL_SIZE	Max Postgres connections held open per worker process (SQLAlchemy pool)	Postgres max_connections	(workers × db_pool_size) + web pools + checkpointer must stay under max_connections
MODULO_MAX_LOCAL_CONCURRENCY	Max concurrent pipeline runs in local execution mode (no E2B sandbox)	Local machine CPU/memory/DB capacity	Keep below what a single host can sustain – local mode runs on your machine, not sandboxes

These four knobs are all operator config. If you came looking for them in the UI, see Why don’t I see pool-size settings in the UI?.

The sizing relationship

Each concurrent run consumes roughly:

• N Postgres connections – its connection from the worker pool plus per-run connections,
• M Redis connections – while job work is in progress,
• One E2B sandbox – for pipeline execution.

The sizing formulas, expressed as budgets that must not be exceeded:

total DB budget     = (workers × SAQ_WORKER_DB_POOL_SIZE)
                      + web-layer pools + checkpointer
                      ≤ Postgres max_connections

total Redis budget  = (workers × SAQ_REDIS_POOL_SIZE) + web-layer connections
                      ≤ Redis maxclients

concurrent sandboxes = product-layer concurrency target
                      ≤ E2B account sandbox quota

Worked example

These are example numbers from farnalabs’ own dogfood deployment – not product defaults. Your values will differ. The point is the process, not the numbers.

Example deployment (farnalabs dogfood, 2026-08-06):

Worked example

Resource	Value
Postgres max_connections	300 (~40 in use)
Redis connected clients	~15
SAQ_REDIS_POOL_SIZE	5
SAQ_WORKER_CONCURRENCY	5
Fly machines	2–5

With 5 Fly machines (one worker process each) and SAQ_REDIS_POOL_SIZE=5, worst-case Redis usage is 5 × 5 = 25 connections – well within a typical Redis maxclients, and consistent with the ~15 clients actually observed in use. On the DB side, even with 5 workers × a modest SAQ_WORKER_DB_POOL_SIZE, the budget lands well below max_connections=300 (observed ~40 in use), leaving generous headroom for the web layer and the checkpointer.

The sizing principle: compute the worst case, verify it against the verified limits of your Postgres and Redis, then set the infra knobs to stay comfortably under them. Only raise the product knobs (concurrency target) when the infra has verified headroom.

Tuning workflow

1. Choose your product concurrency target – how many concurrent runs your pipelines and org sandbox cap should allow.
2. Verify your infrastructure supports it – check SHOW max_connections; on Postgres, Redis CLIENT LIST or the dashboard’s maxclients setting, and the E2B dashboard’s sandbox quota. Use the observed baseline as your starting point, not the hard limit.
3. Set the infra knobs below the verified limits – apply the formulas above with headroom for the web layer and checkpointer.
4. Raise the product knobs only if your infra has headroom – verify with the formulas before exposing more concurrency to customers.
5. Monitor – watch claim_count alerts (runs backing up), lock_timeout errors in run results, and connection saturation on Postgres/Redis. Any of these means the infra layer is undersized relative to the product layer.

FAQ

Why don’t I see pool-size settings in the UI?

Because they are operator environment configuration, not product settings. SAQ_WORKER_CONCURRENCY, SAQ_REDIS_POOL_SIZE, SAQ_WORKER_DB_POOL_SIZE, and MODULO_MAX_LOCAL_CONCURRENCY describe your deployment’s Postgres/Redis capacity and must be tuned by whoever runs the server. The product layer (Pipeline.max_concurrent_runs, the org sandbox cap) is what your users interact with – and what they set to match their own infrastructure.

What happens if I exceed my infrastructure capacity?

It depends on which layer hits the limit first:

• Product layer first – runs are deferred while a capacity slot is unavailable, and fail cleanly with error_code="lock_timeout" once lock_wait_timeout_seconds elapses. This is the designed, graceful behaviour.
• Infrastructure layer first – you get connection errors from Postgres or Redis (too many connections, refused connections) and sandbox quota exhaustion. These are ungraceful and affect the whole deployment, not just one run.

If you are seeing infrastructure-layer errors, scale down the product knobs or scale up the infrastructure – the two layers have drifted out of sync.