Limits

Layer is limited by certain constraints of the underlying components we ship with. We will lift these as demand increases.

• Single-node Aerospike. We enforce this for simplicity and also believe that a single large NVMe drive offers enough storage for almost every dataset.
• ~4,090 Turbopuffer namespaces. We use Aerospike sets for logical separation of data, which are limited by the Aerospike Community Edition AGPL license.
• ~3 TB cache size. Another limitation of the Aerospike license.
• 10,000 distinct values per scan facet field. Pre-computed snapshot scans cap each facet field’s cardinality. If a field exceeds the cap, it is noted in fields_skipped[] rather than fields[], so readers can treat every emitted field as complete. See snapshots.

No limits

These have no enforced ceiling, but practical limits exist and will surface under load.

• CRD instances (Index, Function, Pipeline, Scaling) — bounded only by the etcd and operator throughput of your Kubernetes cluster.
• Snapshot history per namespace — durable in S3, accumulates indefinitely; bounded by object storage cost.
• Search history retention — accumulates indefinitely in S3; no automatic expiry.
• Clickstream event volume — accumulates indefinitely in S3; no automatic expiry.
• UDF concurrency per function — KEDA scales replicas to match queue depth, bounded by your cluster’s capacity.
• Pipeline queue depth — pipeline queues, including chunked document queues, store document IDs and chunk ID lists in S3 manifests and keep only segment state and counters in Postgres.
• Document size and attribute count — bounded by Turbopuffer and Aerospike record limits, not by Layer.