MessageWeave/cloudflare-worker

MessageWeave Cloudflare Worker load balancer

中文版本 / Chinese version → README.zh-CN.md

This is the canonical English version. The Chinese version is README.zh-CN.md.

One public HTTPS entry point in front of one or more MessageWeave backends, with bounded multi-origin failover (HA/LB sub-project). Pass-through model: the Worker does not parse business payloads and forwards the request as-is to the backend https origins; only timeout / 5xx triggers a bounded failover to another instance.

This component is a load balancer for the fixed set of routes MessageWeave needs behind one URL: the route set is fixed (19 paths), the backend origins are fixed at deploy time and https-only, and every unknown path gets a 404. It only forwards requests to those fixed backends and fails over between them. It does not accept an arbitrary target host and provides no general-purpose traffic-relay capability.

Why it exists. Cloudflare's Load Balancer product is not available on the free plan, so this is the way a free-plan account gets multi-origin load sharing and failover in front of its MessageWeave backends. That is the whole reason for it, and the load balancer is optional: the backend runs identically without it.

Why it is not a security feature. There is no intent here to hide the backends and no intent to harden them. The origins stay directly reachable with or without the load balancer and no access control is added at the edge, so the route safelist is not a firewall and nothing in this document is meant to be read as protection.

Design basis: docs/design.md §11.3 (NFR-HA-MULTI-INSTANCE) and docs/charter.md §8. Security baseline and prohibitions: docs/charter.md §3, docs/charter.md §5; stable-ID registry docs/charter.md §8.

1. What it is, and what it is not

• Forward only. Method, headers (including auth headers) and body are forwarded as-is. The Worker performs no authentication of its own and rewrites nothing.
• Secretless. No Telegram bot token, no JMAP password, no session secret. It forwards whatever auth header it is given and checks none of them. All backends must instead share the same set of SAF-AUTH-* secrets — RECONCILE_TOKEN, TG_WEBHOOK_SECRET and the encrypted business config — because a callback can land on any instance and no instance can prove which one was asked (C-LB-SHARED-SECRETS). Per-instance secrets mean a random 401.
• Not a security layer. It is a load balancer, not an access-control boundary. The backend origins are directly reachable with or without it, and nothing here hides or hardens them. The safelist's fail-closed answer on unknown paths is what makes the balancer well-defined for a fixed set of paths, not a protection to be relied on.
• No origin discovery. Origins come from the deploy-time origin list only.
• Zero runtime dependencies. Pure ES2022 plus the platform fetch, Headers, Request, Response and URL. wrangler is a devDependency, used only by check, deploy and dev.
• Zero state. No Redis, no JMAP, no database (C-NO-DB, C-REDIS-ONLY-STATE). The only mutable state is an in-isolate health probe cache.
• No long-lived connections (C-NO-LONG-CONN). Pure request-response. The request body is read once as arrayBuffer and re-attached on every attempt, so streaming is never required.
• Not a queue. There is no retry beyond the bounded attempts below. When every origin fails the load balancer answers 503, and Telegram / Stalwart redelivery is what makes up for it.

2. Directory layout

file	role
wrangler.toml	name, main, compatibility flags, [vars]
package.json	check / test / deploy / dev scripts only, no runtime dependencies
src/index.js	entry point: HTTPS redirect, route dispatch, safelist, health aggregate
src/backends.js	origin parsing and validation, route safelist
src/lb.js	pass-through forwarding and bounded failover
src/health.js	TTL-cached health aggregate probe
test/*.test.js	38 tests in four files, all offline

3. Pass-through semantics

• Method, headers and body go out unchanged; the backend response status, body and headers are returned as-is.
• The only header the load balancer adds is best-effort observability: x-lb-backend, naming the origin that answered. The write sits behind a try/catch. A cross-origin response that carries no access-control-allow-* header comes back with an immutable header guard, and headers.set throws on those. An observability write that throws must never mask a healthy 2xx as a 503, so the header is dropped silently when it cannot be written. That exact defect was the 2026-09-30 production incident.
• GET and HEAD never forward a body: a bodyless upstream response is returned as it is, and no Content-Length is recomputed.
• Backend 3xx redirects are followed by the load balancer itself (redirect: "follow"), so a redirecting origin never hands the browser a second hop.
• Logs and error strings carry method, path, origin and failure class only. Never headers, body, auth secrets or App Password (SAF-LOG-PURITY).

4. Route safelist

Every inbound HTTP request gets a permanent 308 redirect to the same HTTPS URL before route validation or forwarding (C-HTTPS-INBOUND). After that the request must hit one of 19 paths and carry a method that path allows; every other combination is refused at the edge.

path	methods
/	GET
/assets/config.js	GET
/assets/styles.css	GET
/api/status	GET
/api/config	GET PUT
/api/business-config	GET PUT
/api/business-config/preflight	POST
/api/admin/session	POST
/api/admin/session/revoke	POST
/api/enabled	GET PUT
/webhook/tg	POST
/push/jmap	POST
/api/push/register	POST
/api/telegram/register-webhook	POST
/api/push/disable	POST
/reconcile	POST
/worker	POST
/healthz	GET
/ready	GET

• Unknown path: 404, body route not forwarded: <path>. The load balancer can therefore never be turned into a jump host for arbitrary backend paths.
• Method mismatch on a known path: 405, body method not allowed for <path>, plus an Allow header.
• Bootstrap (POST /api/bootstrap) and the remote diagnostics face (/debug/*) are deliberately absent, so both stay origin-only.
• /healthz-worker is not in the safelist at all: the load balancer serves it itself, on GET only.

5. Timeout and retry budget

variable	required	default	applies to
BACKEND_ORIGINS_JSON	yes	—	all routes
LB_REQUEST_TIMEOUT_MS	no	10000	every route except the two below, and the §7.2 probe
LB_MAX_ATTEMPTS	no	2	attempts per request
LB_RECONCILE_TIMEOUT_MS	no	320000	POST /reconcile only
LB_WORKER_TIMEOUT_MS	no	300000	POST /worker only
LB_HEALTH_TTL_MS	no	30000	/healthz-worker probe cache only
LB_VERSION	no	unknown	/healthz-worker version field

Integer env values are parsed with Math.floor. A non-finite or non-positive value is ignored and the default applies, so a typo cannot switch the budget off.

/reconcile and /worker are single-attempt routes with per-route timeout overrides. Both are long synchronous backend jobs: at the 10 s global timeout the load balancer would declare them failed and fail over to a second instance, but that instance answers /reconcile with 409 immediately (the cluster-wide lock:reconcile starts with a 300 s lease, renewed by heartbeat) and a second /worker would merely drain the same batch twice. They therefore get LB_RECONCILE_TIMEOUT_MS (default 320 s, headroom over the 300 s lease plus heartbeats) and LB_WORKER_TIMEOUT_MS (default 300 s, equal to the backend per-event floor SINGLE_EVENT_CEILING_FLOOR_MS), both with max_attempts = 1.

LB_WORKER_TIMEOUT_MS must grow if the /worker batch grows: the backend batch default and its hard cap are both 10.

6. Bounded failover

• Each request tries max(1, min(LB_MAX_ATTEMPTS, origins.length)) origins. The default of 2 is one first attempt plus one failover.
• Only a timeout or a 5xx response moves on to the next origin. 2xx, 3xx and 4xx are answers and are returned immediately.
• A timeout or transport failure counts as retryable when the error name is AbortError, or when the message, or the message of a cause further down the chain, matches one of: fetch failed, network error, socket hang up, ECONNREFUSED, ECONNRESET, EAI_AGAIN, ENOTFOUND, ETIMEDOUT, EHOSTUNREACH, ENETUNREACH. Anything else fails fast.
• With more than one origin and more than one attempt, the starting origin is chosen with rng (defaulting to Math.random) so load spreads over the healthy instances. A single origin degrades to a fixed order.
• Every origin fails: 503 with status text All Backends Unavailable and body all backends failed: <last reason>. Telegram and Stalwart then redeliver, which is why nothing is lost.

The health cache feeds only the /healthz-worker aggregate view. Forwarding does not filter by health; today, failover is the safety net.

7. Health endpoints

The two endpoints deliberately do not overlap, so neither can hide the other.

7.1 GET /healthz — passthrough

Forwarded like any other safelisted route. The response is the backend's own liveness envelope, which is exactly why it can read healthy while a different origin is down.

7.2 GET /healthz-worker — load balancer aggregate

Answered by the load balancer itself. It never touches the forwarded route set.

• Probes each origin's /healthz with the same request timeout as forwarding (LB_REQUEST_TIMEOUT_MS, default 10 s). Redirects are followed, so an origin that answers with a 3xx still counts as up.
• up is true when the origin answered with a status below 500. A probe failure records up: false with status: null.
• Results are cached per origin for LB_HEALTH_TTL_MS (default 30 s) in an in-isolate map, to keep probe traffic bounded. A cache miss probes and stores; a fresh entry is served without another probe.
• Response body: {status, version, available, total, backends: [{origin, up, status}]}.
  • at least one origin up: HTTP 200, status: "ok"
  • every origin down: HTTP 503, status: "down"
  • no origins configured: HTTP 200, status: "no-backends"
  • version is LB_VERSION, defaulting to the literal string unknown
• Any method other than GET: 405 with Allow: GET.
• The load balancer does not proxy Redis or JMAP readiness. End-to-end backend readiness (ARCH-READY-BASELINE: config completeness, Redis reachable, JMAP session, Telegram getMe) stays on the backend's /ready, which is passed through.

8. Callback registration

The SPA registers callback URLs against one origin (C-LB-SINGLE-REG-URL). Origin only, no path:

• Load balancer enabled: the Worker URL, for example https://lb.example.
• Load balancer disabled: a backend origin, for example https://a.example.

Whichever origin you register is where /webhook/tg and /push/jmap land. All three registration endpoints are safelisted and pass through the load balancer, so registration works through either front door. Registration is idempotent, and registering a new address retires the previous subscription — that is how you switch fronts.

9. Configuration

• Defaults live in wrangler.toml [vars] as a deliberate plaintext exception: the current value is only publicly reachable https origins, which is not a credential. The moment you need an internal address, a URL with embedded credentials, or a hostname that should not be public, move it to npx wrangler secret put BACKEND_ORIGINS_JSON and delete the line from [vars]. Git history is irreversible, so deleting it does not un-publish it.
• BACKEND_ORIGINS_JSON is a JSON array of plain strings. Object shapes such as {"url": "..."} are rejected: parseBackendOrigins throws, and because the config is re-parsed on every request the failure shows up as 503 on every request rather than at boot. The body is misconfigured backends. An empty list throws the same way, before anything is forwarded.
• Each entry must be an https origin only: no http, no path, no query, no fragment, no embedded credentials (C-HTTPS-INBOUND).
• wrangler.toml also pins name = "messageweave-lb", main = "src/index.js", compatibility_date = "2025-01-01" and compatibility_flags = ["nodejs_compat_v2"]. nodejs_compat_v2 is what lets the test files import node:test and node:assert/strict. The runtime code imports no node module.
• Comments in wrangler.toml must stay single-line # comments. TOML has no block comment, so a JSDoc /** */ block makes wrangler fail to parse the file at all.

10. Local and CI verification

cd cloudflare-worker

npm test       # 38 tests, four files, fully offline
npm run check  # node --check on sources and tests, then wrangler deploy --dry-run

npm test is node --test, which picks up test/*.test.js: test/index.test.js 15, test/lb.test.js 12, test/backends.test.js 6, test/health.test.js 5. There is no install step for the tests — the Worker has no runtime dependencies, and the test files import only node:test and node:assert/strict.

Two traps:

• node --test test/ (a directory) exits 0 after running zero tests on some Node versions. Always pass the file glob.
• A default CI install step of bun install ignores the npm lockfile: it re-resolves and can move wrangler across major versions. Add an explicit npm ci before the test step if you need the pinned wrangler.

11. Deploy

cd cloudflare-worker
npm install --no-audit --no-fund
npm run deploy
curl https://<your-worker>.workers.dev/healthz-worker

npm run check includes wrangler deploy --dry-run --outdir=.build-check, which bundles the four sources and reports the artifact size before anything is published. The load balancer is deployed from push, so a curl of /healthz-worker and a compare of its version is the acceptance check. Bump LB_VERSION in the same commit as any LB logic change, so you can tell which build answered.

Two production incidents worth keeping in mind:

• 2026-09-30: a build wrote x-lb-backend onto proxied responses. Cross-origin responses without access-control-allow-* headers have an immutable header guard, so headers.set threw and the surrounding handler logged a healthy backend as unavailable and burned its retries — the load balancer returned 503 while GET / actually returned 200. /healthz-worker did not go through that code path, which is exactly what hid the outage. Fixed by making the header write best-effort.
• Object-shaped origin lists fail silently at parse time. parseBackendOrigins throws and you get 503 misconfigured backends on every request. Keep weight and any other per-origin tuning out of the list: nothing reads them.

12. Design invariants

• SAF-LB-PASSTHRU — no auth, header or body rewriting on the path
• C-HTTPS-INBOUND — https-only origins, embedded credentials and paths rejected, HTTP gets a 308
• C-LB-SINGLE-REG-URL — one callback origin, unknown paths 404
• C-LB-SHARED-SECRETS — every backend carries identical business credentials
• MOD-HEALTH-AGG — TTL-cached aggregate probe owned by the load balancer
• C-NO-DB, C-REDIS-ONLY-STATE — no database or Redis access from the load balancer
• C-NO-LONG-CONN — request-response only
• ARCH-READY-BASELINE — end-to-end readiness stays on the backend's /ready
• SAF-LOG-PURITY — method, path, origin and failure class only
• ARCH-LB-WORKER — the load balancer is the only edge component
• NFR-HA-MULTI-INSTANCE — multi-origin failover is how HA is realised

References

• docs/reference.md §9.5 — load balancer deployment, configuration and rollout
• docs/reference.md §4 — load balancer versus backend route matrix
• docs/reference.md §9.2 — backend configuration and secrets
• docs/reference.md §9.4 — health and release checks
• docs/design.md §11.3 — HA and multi-instance design
• docs/charter.md §3, docs/charter.md §5, docs/charter.md §8 — security baseline, prohibitions, stable-ID registry