amadan
server: mesh_peer_boxes must exclude this box's own stats entry (#847)
Found live during the mesh flip re-attempt (box A, 2026-07-13): once a box's own mesh is on, it self-publishes its own stats snapshot to the same S3 prefix mesh_peer_boxes reads back (meshstats.go's startStatsReporter runs on every mesh-enabled box, not just peers). Box A's build_fleet already counts itself as "the local box" via its own OS reads earlier in the function — without excluding its own S3 entry, mesh_peer_boxes double-counted it as a "peer" too. The Fleet page briefly showed hosts.count 3 (one real duplicate label) instead of 2 as a result. Adds a small b64url_decode helper (reversing encodeHeartbeatKey's encoding, same as mesh.go's decodeHeartbeatKey) and skips any stats key that decodes to this box's own LCHAT_MESH_SELF. Harmless for a box whose own mesh isn't on (LCHAT_MESH_SELF unset, self stays empty, nothing skipped) and for the dry-run path (unaffected — self-exclusion only matters once box_peer listing runs against real S3). Real customer traffic was never affected — this is purely the Fleet status page's own aggregation, not the routing/placement path.
Co-authored-by: Claude <noreply@anthropic.com>
1 file changed,
+24
−1
deploy/status-publish.sh+24 −1
diff --git a/deploy/status-publish.sh b/deploy/status-publish.sh| index aceeafe..47ce8eb 100755 |
| --- a/deploy/status-publish.sh |
| +++ b/deploy/status-publish.sh |
| @@ -283,6 +283,20 @@ env_str() { |
| grep -E "^$2=" "$1" 2>/dev/null | head -n1 | cut -d= -f2- || true |
| } |
| +# b64url_decode reverses encodeHeartbeatKey (mesh.go): url-safe alphabet, |
| +# padding stripped — re-add it (len mod 4 tells you how much) before handing |
| +# off to the standard decoder. Used by mesh_peer_boxes to recognize (and |
| +# skip) THIS box's own entry in the stats prefix. |
| +b64url_decode() { |
| + local s |
| + s=$(printf '%s' "$1" | tr -- '-_' '+/') |
| + case $(( ${#s} % 4 )) in |
| + 2) s="${s}==" ;; |
| + 3) s="${s}=" ;; |
| + esac |
| + printf '%s' "$s" | base64 -d 2>/dev/null |
| +} |
| + |
| # disk_free_gb — whole GB free on the data disk (`df -Pk`: POSIX, and avail |
| # matches controld's Statfs Bavail). Empty on failure — an unknown contributes |
| # nothing to the band, never a guess. |
| @@ -358,12 +372,20 @@ mesh_peer_boxes() { |
| return |
| fi |
| - local bucket region prefix mesh_prefix |
| + local bucket region prefix mesh_prefix self |
| bucket=$(env_str "$ROUTER_ENV" LCHAT_HIBERNATE_BUCKET) |
| prefix=$(env_str "$ROUTER_ENV" LCHAT_HIBERNATE_PREFIX) |
| if [ -z "$bucket" ] || [ -z "$prefix" ]; then echo '[]'; return; fi |
| region=$(env_str "$ROUTER_ENV" LCHAT_HIBERNATE_REGION) |
| region=${region:-eu-west-1} |
| + # This box's own mesh identity, if its OWN mesh is on (LCHAT_MESH_SELF set) |
| + # — skipped below. Without this, a box whose mesh is on double-counts |
| + # itself the moment it starts self-publishing to this same stats channel: |
| + # once as "the local box" (this script's own OS reads, elsewhere in |
| + # build_fleet) and again as a "peer" read back from S3. Found live |
| + # 2026-07-13, box A's flip: the Fleet page briefly showed hosts.count 3 |
| + # (one real duplicate) instead of 2. |
| + self=$(env_str "$ROUTER_ENV" LCHAT_MESH_SELF) |
| # Must match hc.meshPrefix() (activator.go) exactly — the membership |
| # heartbeat lives at "<prefix>/mesh/members/<peer>", and statsKey |
| # (internal/activator/heartbeat.go) appends "-stats" to that WHOLE path, |
| @@ -401,6 +423,7 @@ mesh_peer_boxes() { |
| key=$(printf '%s' "$line" | awk '{print $4}') |
| [ -n "$key" ] || continue |
| [ "$date_part $time_part" '>' "$cutoff" ] || continue # stale — treat like a dead mesh member |
| + if [ -n "$self" ] && [ "$(b64url_decode "$key")" = "$self" ]; then continue; fi |
| tmp="$TMP/mesh-stat-$(printf '%s' "$key" | tr '/' '_').json" |
| aws s3 cp "s3://$bucket/${mesh_prefix}-stats/$key" "$tmp" --region "$region" >/dev/null 2>&1 || continue |
| body=$(cat "$tmp" 2>/dev/null) || continue |