Skip to main content

​
Why a pool

A fresh microVM cold-boot from scratch (kernel + init + agent-ready) takes around 1.4 s. A snapshot-restore is ~12 ms. Agents want sandboxes in under 100 ms server-side, every time. The only way is to pre-boot them and hand them out.

​
The shape

Warm pool architecture
At startup, each region pre-boots a target number of VMs from a seed snapshot and keeps them in an idle queue. create() pops one and returns instantly; a background refill tops the queue back up.

​
Lifecycle

1

Startup

The pool manager launches N boot tasks in parallel. Each task restores a VM from the seed snapshot (~12 ms per VM).
2

Steady state

create() pops from the head of the idle queue and returns instantly — ~6 ms server-side (the actual pop_front + DB write), ~187 ms end-to-end p50 including network round-trip from a US laptop (via api.podflare.ai, SDK 0.0.20). The manager is woken to refill.
3

Burst

If the pool empties, new create() calls fall back to a snapshot-backed on-demand boot — typically 150–500 ms server- side, well under the original 1.4 s cold-boot. The pool refills in the background at ~12 ms per VM. A 120-sandbox refill takes under 30 s wall-clock.

​
Choosing the pool size

​
PODFLARE_FC_POOL_SIZE
int
default:"120"
How many idle VMs the manager tries to keep ready. Production default is 120 on each US/SG region (96 GB hosts) and 64 on EU (62 GB host). Set this close to your peak concurrent-create rate on the host. Each idle warm VM consumes only ~40 MB RSS thanks to the Pod runtime’s lazy snapshot restore — the nominal RAM allocation doesn’t get committed until the guest touches a page.
  • Too small → bursts hit on-demand boot fallback (slower hot path).
  • Too large → committed RAM grows past your host’s headroom; leave ~40 % free for on-demand customer creates and Space resumes.
Capacity rule of thumb: pool size up to ~50 % of the host’s RAM in 1 GB units (e.g. 120 warm × 40 MB ≈ 5 GB committed on a 96 GB host, leaves 88 GB headroom).

​
Observability

Each pool event is logged with structured fields. The hostd pool stats line emits per minute per template:
pool_hit_rate_bp is in basis points (9793 = 97.93 %). /v1/capacity exposes the same counters as JSON for the edge router and /status page to consume.