carlos / platform Public

edge: hop XFF passes through verbatim; strip a stray hop signature

Two review findings on Task 3, both in this task's own surface.

1. setForwarded appended this edge's peer to a verified hop's inbound
   X-Forwarded-For, producing "<client>, <sending edge EIP>". That
   breaks last-element client extraction, which is not a guess about
   tenant frameworks but this repo's own documented contract:
   internal/console/authcli.go's clientIP takes the LAST element as the
   caller ("the edge appends its own view of the peer with
   SetXForwarded, so everything before it may have been supplied by the
   client"). Every backhauled console request would have rate-limited
   as one bucket per region, and tenant apps' rightmost-public
   heuristics would have read infrastructure as the caller.

   SetXForwarded still runs first and still sets X-Forwarded-Host and
   -Proto (both correct as computed at this edge); on a verified hop its
   X-Forwarded-For is then REPLACED by the inbound chain verbatim, so a
   backhauled request is indistinguishable from a local one at the app
   and the last element is still the client. The backhaul leg is
   deliberately invisible in that header.

2. acceptHop returned before either Del when X-Carlos-Hop-Auth arrived
   WITHOUT X-Carlos-Hop, contradicting its own "deleted in ALL hop
   cases" invariant and letting a client's stray signature reach the
   tenant app. An app that reflects its request headers would then
   launder edge vocabulary back onto the wire — the exact laundering
   scrubInboundPurgeHeader deletes both purge headers unconditionally to
   prevent (purge_header.go). The fast path now deletes a stray auth
   header, keeping its single Get and touching the key only when one is
   actually present.

Both changes are mutation-checked: reverting either fails its test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Paul Campbell pushed by paul@keymail.dev 67c20343563f561e8adfee6e9234a1675ad92ba9 parent 42844ae
2 files changed, +74 −27
  • internal/edge/homemap.go +52 −25
  • internal/edge/homemap_test.go +22 −2
diff --git a/internal/edge/homemap.go b/internal/edge/homemap.go
index 7ea68def7..14ead04e0 100644
--- a/internal/edge/homemap.go
+++ b/internal/edge/homemap.go
@@ -37,14 +37,18 @@ package edge
// was, which is why verification gates exactly one thing: whether the
// inbound X-Forwarded-For is believed.
//
-// - XFF is trusted as an APPENDED element, never as a replacement. On a
-// verified hop the inbound chain is carried onto the outbound request and
-// the sending edge's own address is appended to it (setForwarded, below),
-// so the app sees "<client>, <sending edge>" — the same shape a
-// single-region app already sees, with one more hop in it. On anything
-// else the chain is discarded outright: an unverified sender is an
-// ordinary client, and an ordinary client has never been allowed to state
-// its own address here.
+// - XFF is trusted as an inbound chain to CARRY, never one to extend. On a
+// verified hop the chain the sending edge built is passed through
+// verbatim (setForwarded, below), so the app sees exactly what a
+// single-region app sees and the LAST element is still the client. The
+// backhaul leg is deliberately invisible in that header: this
+// deployment's own consumers take the last element as the caller
+// (internal/console/authcli.go's clientIP, and every tenant framework's
+// rightmost-public heuristic), so appending the sibling edge would make
+// every backhauled request look like it came from infrastructure. On
+// anything but a verified hop the chain is discarded outright: an
+// unverified sender is an ordinary client, and an ordinary client has
+// never been allowed to state its own address here.
//
// One hop, ever, is STRUCTURAL and not a policy check: a verified hop that
// names a host this box does not serve leaves through staleHopRefusal (503 +
@@ -488,6 +492,19 @@ func hopVerified(ctx context.Context) bool {
// signer and a verifier can never disagree (mesh.go).
func (e *Edge) acceptHop(r *http.Request, host string) (*http.Request, bool) {
if r.Header.Get(hopHeader) == "" {
+ // A signature with no marker beside it is not a hop — but it is still
+ // this edge's vocabulary, and the invariant above is "deleted in ALL
+ // hop cases" without an exception for the half a client made up.
+ // scrubInboundPurgeHeader deletes both purge headers unconditionally
+ // for exactly this reason (purge_header.go): an app that reflects what
+ // it was sent turns a client-supplied edge header into an
+ // app-DECLARED one, and no tenant's reflection bug should be able to
+ // put hop vocabulary back on the wire. The Get stays the fast path's
+ // single lookup — the Del only happens on a request that actually
+ // carries the stray header, which is never, in the ordinary case.
+ if r.Header.Get(hopAuthHeader) != "" {
+ r.Header.Del(hopAuthHeader)
+ }
return r, false
}
auth := r.Header.Get(hopAuthHeader)
@@ -540,25 +557,35 @@ func staleHopRefusal(w http.ResponseWriter) {
// pr.SetXForwarded(), and it exists for one narrow case.
//
// ReverseProxy's Rewrite path deletes Forwarded and X-Forwarded-* from the
-// outbound request before Rewrite runs, so SetXForwarded's documented "append
-// to an existing value" behaviour never fires: an app behind this edge sees
-// exactly one address, the peer that dialled us. That is right for a client
-// and wrong for a verified hop, where the peer that dialled us is a sibling
-// edge and the address the app actually wants is the one that edge recorded.
+// outbound request before Rewrite runs, so SetXForwarded always writes a
+// single address: the peer that dialled us. That is right for a client and
+// wrong for a verified hop, where the peer that dialled us is a sibling edge
+// and the address the app wants is the one that edge already recorded.
//
-// So on a verified hop — and only there — the inbound chain is copied onto the
-// outbound request first, and SetXForwarded then appends the sending edge to
-// it. Multiple inbound X-Forwarded-For lines are joined, which is what
-// list-valued header semantics already say they mean (RFC 9110 §5.3).
+// So on a verified hop — and only there — SetXForwarded's answer is REPLACED
+// by the inbound chain, verbatim. Not appended to: the last element has to
+// stay the client. This deployment's own consumer says so in as many words
+// (internal/console/authcli.go's clientIP: "the LAST element is the one to
+// take"), and tenant frameworks' rightmost-public heuristics agree — a chain
+// of "<client>, <sending edge EIP>" would rate-limit every backhauled console
+// request as one bucket per region and show tenant apps infrastructure in
+// place of their caller. Passing the chain through unchanged means a
+// backhauled request is indistinguishable from a local one at the app, which
+// is the whole promise of the hop. Multiple inbound X-Forwarded-For lines are
+// joined, which is what list-valued header semantics already say they mean
+// (RFC 9110 §5.3).
//
-// X-Forwarded-Host and X-Forwarded-Proto need no such restore: the hop carries
-// the original Host header and is itself TLS, so SetXForwarded recomputes both
-// correctly at this edge.
+// SetXForwarded still runs first, and is still what sets X-Forwarded-Host and
+// X-Forwarded-Proto: the hop carries the original Host header and is itself
+// TLS, so both are correct as computed here and need no restore. It touches
+// only pr.Out, so reading the inbound chain after it is the same read as
+// before it.
func setForwarded(pr *httputil.ProxyRequest) {
- if hopVerified(pr.In.Context()) {
- if chain := pr.In.Header.Values("X-Forwarded-For"); len(chain) > 0 {
- pr.Out.Header.Set("X-Forwarded-For", strings.Join(chain, ", "))
- }
- }
pr.SetXForwarded()
+ if !hopVerified(pr.In.Context()) {
+ return
+ }
+ if chain := pr.In.Header.Values("X-Forwarded-For"); len(chain) > 0 {
+ pr.Out.Header.Set("X-Forwarded-For", strings.Join(chain, ", "))
+ }
}
diff --git a/internal/edge/homemap_test.go b/internal/edge/homemap_test.go
index b490a4522..3fe510c9c 100644
--- a/internal/edge/homemap_test.go
+++ b/internal/edge/homemap_test.go
@@ -326,8 +326,13 @@ func TestVerifiedHopIsServedLocallyAndHonoursTheInboundXFF(t *testing.T) {
t.Fatalf("status = %d, want 200", resp.StatusCode)
}
body := readBody(t, resp)
- if !strings.HasPrefix(body, "xff=203.0.113.9, ") {
- t.Fatalf("body = %q — a verified hop's inbound XFF must be kept and the sending edge appended to it", body)
+ // Verbatim, with NO element appended: the last element must still be the
+ // client. internal/console/authcli.go's clientIP takes the last element as
+ // the caller and every tenant framework's rightmost-public heuristic does
+ // the same, so appending this edge's peer (the sibling edge) would make a
+ // backhauled request look as though infrastructure called it.
+ if !strings.HasPrefix(body, "xff=203.0.113.9|") {
+ t.Fatalf("body = %q — a verified hop's inbound XFF must pass through verbatim, last element still the client", body)
}
// The hop's own trust headers are the edge's vocabulary, never the app's.
if !strings.Contains(body, "|hop=|auth=") {
@@ -431,6 +436,21 @@ func TestAcceptHopStripsEveryForwardedHeaderOnFailedVerification(t *testing.T) {
if hopVerified(r.Context()) {
t.Fatal("an unverified request must not carry the hop mark")
}
+
+ // The half-formed case: a signature with no X-Carlos-Hop beside it. It is
+ // not a hop and must not be treated as one — but the header is still this
+ // edge's vocabulary, and leaving it on the request would let an app that
+ // reflects its request headers launder it back onto the wire (the
+ // scrubInboundPurgeHeader precedent, purge_header.go).
+ stray := httptest.NewRequest("GET", "http://app.test/", nil)
+ stray.Header.Set(hopAuthHeader, "0:deadbeef")
+ stray, verified = e.acceptHop(stray, "app.test")
+ if verified {
+ t.Fatal("an auth header with no hop marker is not a hop")
+ }
+ if got := stray.Header.Get(hopAuthHeader); got != "" {
+ t.Fatalf("%s survived without a hop marker as %q — hop vocabulary must never reach the app", hopAuthHeader, got)
+ }
}
// The MAC is over the host, so a signature minted for one hop cannot be