Shapes and cache tiers

An instance has a shape and a cache. Both are chosen at creation and can be changed later while the instance is stopped.

Shapes

Shape	Machines	CPU	Durability	Use it for
ramp	1	Fly shared CPU	Every write is an S3 conditional put before it is acknowledged.	Development, single-writer workloads, anything that tolerates a short outage when the Machine is replaced.
launch	3	Fly performance CPU	Each Machine runs the embedded Bitr write-ahead log; a write commits at a quorum of two and is archived to S3 within a second.	Production workloads that need to keep serving while a Machine is replaced.

Every Machine in a launch instance is an API node behind the same URL. The engine owns stream placement between them; nothing pins a client to a node.

SQL across nodes

Each table on a launch cluster is owned by one of the three nodes. A statement whose tables all have the same owner is forwarded to that owner and runs there. A statement that spans tables owned by different nodes works too: the node that receives it gathers the rows it does not own from their owners and runs the query locally.

A gather is refused above a million rows in a single table:

stream <name> has more than 1000000 rows; query it on its owner rather than joining it across members

The limit is on how many rows one table has to move, not on how large a table may be or how many a query may name. The remedy is to stop moving that table’s rows: query it on its own, or alongside only tables its own node owns, and the cluster forwards the whole query to its owner so nothing is gathered. If you need the join, narrow the large table first so fewer than a million of its rows take part.

Cache tiers

The cache tier sets each Machine’s memory and disk; the shape decides the CPU kind. A tier is one Machine and one disk per shape: the memory is the Machine’s, and the disk is the NVMe volume it mounts, created at exactly that size, four times the memory. The engine is told both numbers as they are and budgets them itself, holding back a small floor of each (256 MiB of memory, plus 128 MiB with Bitr, and 512 MiB of disk) for its own working set. A ramp Machine has 2 GiB per shared vCPU; a launch Machine has 4 GiB per performance vCPU and is one size up.

Shape	Cache	Machine	Disk	Per hour
ramp	small	shared-cpu-1x, 2 GiB	8 GB	$0.054, serverless
ramp	medium	shared-cpu-2x, 4 GiB	16 GB	$0.108
ramp	large	shared-cpu-4x, 8 GiB	32 GB	$0.204
ramp	xlarge	shared-cpu-8x, 16 GiB	64 GB	$0.410
launch	small	performance-2x, 8 GiB	32 GB	$1.095
launch	medium	performance-4x, 16 GiB	64 GB	$2.053
launch	large	performance-8x, 32 GiB	128 GB	$4.108
launch	xlarge	performance-16x, 64 GiB	256 GB	$8.218

The cache holds the working set: recently written fragments, indexes and whatever queries touch. It is not a storage limit. Tables can be larger than the disk; colder data is read from S3 on demand.

How many tables a tier keeps open

A tier’s practical limit is usually how many tables it can keep open at once, not how much data it holds. Opening a table leases memory for the whole time it stays open:

48 MiB  +  100,000 × (dimensions × 4 + 128) bytes  per vector column

A table with no vector column costs only the 48 MiB. The vector part is the in-memory graph, sized for the memtable’s row capacity, and it is charged per vector column.

A 768-dimension table is the common case:

100,000 × (768 × 4 + 128) = 320,000,000 bytes = 305 MiB
305 MiB + 48 MiB = 353 MiB

A ramp / small instance has 2 GiB. The engine holds back its own floor and shares what is left with memtables, inserts and compaction, so roughly four 768-dimension tables fit open at once.

Opening one more than fits is refused, and the error names the table, how much memory it needed, how much of the budget was left and how much was already held. Nothing is evicted to make room and the cache is not degraded to fit the extra table: the open is simply refused.

The refusal is temporary

A table that has gone five minutes without being touched is checkpointed, closed, and its memory released. The next use reopens it. That costs a reopen and never data.

So when an open is refused:

• Retry after a few minutes in which the other tables are idle.
• Keep fewer tables open at a time.
• Move up a cache tier.
• Drop a table you no longer need, which frees its memory at once rather than in five minutes.

Any query or insert against a table resets its idle clock. A workload that touches every table every minute never lets one go idle, so nothing is ever released and the refusal will not clear on its own — that one has to be fixed by touching fewer tables or by moving up a tier.

On a launch cluster a node only opens the tables it owns, so the three nodes each lease against their own tier’s memory and the cluster keeps about three times as many tables open as one node would.

Serverless: ramp small

ramp / small is the one tier that is not always on. Fly suspends its Machine after a few idle minutes, taking a snapshot of its memory, and starts it again on the next request: the engine resumes where it was, and the first request after a wake pays the resume. The instance stays running throughout; its record shows the Machine as suspended while it is asleep. Nothing it spends suspended is billed, and it is charged at least $5 a month. Every other shape and tier stays on, and bills, from start to stop.

Stop, start, resize and delete work the same on every tier: an explicit stop still drains the Machine and deletes it and its volume.

Choosing

• Start on ramp / small. It is the cheapest instance, sleeps when idle, and loses nothing on a drained stop, so moving up later is a stop, a resize and a start.
• Pick launch when a Machine replacement must not interrupt writers, or when the workload wants dedicated performance CPUs. The move is permanent: a launch instance cannot be resized back to ramp, only to another cache tier.
• Pick a larger cache when queries spill to S3 more than you like, when your hot tables no longer fit on the disk, or when you need more tables open at once than the tier can lease.

An instance is billed per second at its shape and cache tier’s hourly rate; see Billing. A launch rate covers all three of its Machines.