eleven / messenger Public

Download this file

1	# 🤖 How the infrastructure compares
2
3	Where the Eleven *infrastructure* sits among prior art, honestly — a
4	sibling of [comparison.md](comparison.md), which compares the product, and
5	[comparison-carbon.md](comparison-carbon.md), which estimates this
6	architecture's footprint at scale.
7	The architecture in question is the
8	[#550 plan](https://github.com/elevenmessenger/messenger/issues/550):
9	one fully-isolated single-binary instance per space, hibernating when idle,
10	state parked in S3 as ciphertext, woken on demand by an activator mesh that
11	places instances by lease + rendezvous hashing — no orchestrator, no
12	control plane.
13
14	The short version: **every mechanism has shipped somewhere, and two systems
15	are nearly the same shape — but nobody has shipped this combination.** The
16	reasons are datable: the load-bearing primitives mostly didn't exist before
17	2021–2024, and everyone big enough to build it had business incentives
18	pointing the other way. The people who *did* build the pattern sold it as a
19	rentable platform instead of putting it inside a product.
20
21	## At a glance
22
23	| | Durable Objects | Sandstorm (2014) | Orleans / Erlang | Fly Machines | **Eleven (#550)** |
24	|---|---|---|---|---|---|
25	| Unit | JS actor + private SQLite | per-document container ("grain") | in-process actor over a shared store | micro-VM | **OS process + own SQLite** |
26	| Wakes on demand / sleeps idle | Yes (incl. WebSocket hibernation) | Yes | Yes (activate/deactivate) | Yes (stop/start) | **Yes** |
27	| Durability layer | Cloudflare's storage | local disk | pluggable shared store | volume (+ Litestream) | **S3, continuous replication** |
28	| Placement | Cloudflare's routing layer | single server | directory service | Fly's scheduler | **S3 lease is truth; HRW hash is the hint — no directory** |
29	| Self-hostable | No | Yes (its whole point) | Yes (a framework) | No | **Yes — same binary as the flagship** |
30	| State is ciphertext to the operator | No | No | No | No | **Yes — the E2E rule** |
31
32	## The nearest neighbours
33
34	**[Cloudflare Durable Objects](https://developers.cloudflare.com/durable-objects/)**
35	is almost isomorphic: a per-entity actor addressed by name, woken on
36	request, hibernating when idle — including WebSocket hibernation, where
37	connections stay open while the object sleeps — each with its
38	[own private SQLite database](https://blog.cloudflare.com/sqlite-in-durable-objects/)
39	(2024). "A chat room per object" is literally Cloudflare's canonical demo,
40	and Cloudflare Workflows V2 proved the density story at tens of thousands
41	of concurrent SQLite-backed instances. What it isn't: self-hostable, a
42	normal OS process, or E2EE — it's a JS runtime inside Cloudflare's control
43	plane. Renting it would put the core mechanic on someone else's platform
44	and fork the self-hosted shape, which is why #550 rejects it by name.
45
46	**[Sandstorm](https://sandstorm.io/) (2014)** was the self-hosted version,
47	a decade too early. Grains — per-document isolated containers, spun up on
48	first request, suspended when idle, capability-secured — are the closest
49	anyone came to "the unit of the system is a tiny personal server that
50	hibernates." The company wound down around 2017 with essentially no
51	revenue: containers were heavy, every app had to be ported into its
52	sandbox, and none of the enablers below existed yet. The instructive coda:
53	its founder, [Kenton Varda](https://github.com/kentonv), went to Cloudflare
54	and built Workers and Durable Objects. The grain idea didn't fail — it got
55	captured into a proprietary platform.
56
57	**[Orleans](https://learn.microsoft.com/en-us/dotnet/orleans/overview) and
58	Erlang/OTP** are the 15–40-year-old theory. Orleans "virtual actors"
59	(Microsoft Research 2011, Halo's backend by 2012) activate on demand,
60	deactivate when idle, and place across a cluster automatically. The
61	difference: actors are code objects over a *shared* store with a directory
62	service; Eleven's unit is a whole isolated process with its own database,
63	and placement needs no directory because the S3 lease is the truth. The
64	activator itself descends from `inetd` (1986) — wake-on-connection is the
65	oldest trick in Unix, and 90s shared PHP hosting was economically
66	process-per-site scale-to-zero.
67
68	**[Fly.io](https://fly.io/)** built the tooling and evangelized the
69	pattern — Machines that stop when idle and wake on request, plus
70	[Litestream](https://fly.io/blog/all-in-on-sqlite-litestream/), whose
71	author it hired to push server-side SQLite — explicitly pitching "one app
72	per customer." Again: rented placement.
73
74	**The S3-as-the-only-disk wave is happening right now** — WarpStream,
75	SlateDB, turbopuffer, [Turso's](https://turso.tech/) database-per-tenant,
76	Litestream-under-workflow-instances. The storage layer of #550 isn't
77	swimming against the current; it's riding a 2024–2026 wave that only became
78	possible when S3 shipped conditional writes.
79
80	**Product-shape neighbours:** [Snikket](https://snikket.org/)
81	(instance-per-family XMPP, invite links) and 37signals'
82	[ONCE Campfire](https://once.com/) (chat sold as one self-hosted unit per
83	group) share the small-instance-per-group philosophy — but neither
84	hibernates, meshes, or offers a planet-scale hosted plane.
85	[Tarsnap](https://www.tarsnap.com/), [Ente](https://ente.io/), and
86	Bitwarden pioneered client-encrypted-blobs-in-a-bucket, the storage half of
87	the trust model.
88
89	## What only the combination gives you
90
91	1. **E2EE as an infrastructure enabler, not a feature.** Everyone else
92	doing hibernation-over-object-storage has to encrypt at rest, manage
93	keys, and trust the bucket. Because an instance's state is already
94	ciphertext by [the one rule](../README.md), parking it in S3 is
95	threat-model-free — the crypto design and the infra design pay for each
96	other. Nobody has built the hibernation pattern *on top of* a
97	ciphertext-only invariant.
98	2. **The self-hosted unit and the planet-scale unit are the same binary in
99	the same shape.** Durable Objects and Fly are platforms you rent;
100	Sandstorm and Snikket were self-hosted with no flagship-scale story.
101	"One box with no S3 is the degenerate case of the mesh" as a design
102	invariant appears to be new.
103	3. **No control plane.** Orleans has a directory, Durable Objects has
104	Cloudflare's routing layer, Kubernetes *is* a control plane.
105	Lease-as-truth + rendezvous hashing keeps placement entirely inside the
106	app — the IaC layer never learns an instance hostname.
107
108	## Why hasn't this been done before?
109
110	**The primitives are datable, and they're recent.**
111
112	| primitive | arrived | what it unlocks |
113	|---|---|---|
114	| S3 conditional writes (`If-None-Match` / `If-Match`) | Aug / Nov 2024 | the lease — a distributed single-writer lock with no DynamoDB/ZooKeeper |
115	| Litestream (v0.5 leases) | 2021 (2025) | continuous SQLite→S3 replication; restore-anywhere |
116	| Wildcard ACME via DNS-01 | 2018 | per-instance TLS at zero marginal cost |
117	| Pure-Go SQLite + `go:embed` | ~2021 | the one-file, no-cgo, no-build-step binary |
118	| Passkeys with PRF | 2023+ | a keyless, stateless home server |
119	| Server-side SQLite respectability | ~2022–2024 | Fly, Tailscale, then Rails 8 defaulting to it |
120
121	Anyone attempting this stack in 2019 would have had to build four missing
122	layers first — which is roughly what killed Sandstorm.
123
124	**The incentive gradient pointed the other way for everyone capable of
125	it.** The standard SaaS playbook wants one big multi-tenant database: it
126	maximizes density and enables everything the business model runs on —
127	search, analytics, growth loops, and (for messengers) the global social
128	graph. E2EE-plus-isolation forecloses all of that *on purpose*. Signal
129	centralized; Meta needs the graph; enterprise vendors sell isolation as
130	expensive "dedicated instances," not free family chats. Nobody's revenue
131	model demanded a million tiny free isolated servers costing fractions of a
132	cent each — which is exactly why the ones who built the mechanics
133	(Cloudflare, Fly) monetized them as infrastructure-for-rent rather than
134	shipping a product on them.
135
136	**And a team-economics reason that's only just dissolving:** a homegrown
137	activator mesh instead of Kubernetes used to be a staffing decision —
138	bespoke placement layers were for Google-shaped orgs. A tiny team choosing
139	"we'll write our own orchestration inside the app binary" is newly sane in
140	the LLM era, which may be the quietest reason this combination is appearing
141	now and not in 2020.
142
143	## Sources
144
145	- [Zero-latency SQLite storage in every Durable Object (Cloudflare)](https://blog.cloudflare.com/sqlite-in-durable-objects/)
146	- [I'm All-In on Server-Side SQLite (Fly)](https://fly.io/blog/all-in-on-sqlite-litestream/)
147	- [Orleans virtual actors (Microsoft)](https://learn.microsoft.com/en-us/dotnet/orleans/overview)
148	- [Sandstorm](https://sandstorm.io/) · [Kenton Varda](https://github.com/kentonv)
149	- [SQLite durable workflows over Litestream](https://byteiota.com/sqlite-durable-workflows-skip-temporal/) · [turbolite — SQLite VFS over S3](https://github.com/russellromney/turbolite)
150	- The #550 plan itself: [home: use an object store (S3) as the only database — planet-scale plan](https://github.com/elevenmessenger/messenger/issues/550)
151
152	*Snapshot as of July 2026 — this field is moving fast; corrections welcome.*
153