-
fix(cluster): intra-cluster metrics tolerate clock skew + log rejections
Release / version (push) Canceled after 0sRelease / frontend (push) Canceled after 0sRelease / linux (amd64) (push) Canceled after 0sRelease / linux (arm64) (push) Canceled after 0sRelease / windows (push) Canceled after 0sRelease / macos (push) Canceled after 0sRelease / release (push) Canceled after 0sBuild and Push Docker Image / build-and-push (push) Canceled after 0sreleased this
2026-06-21 00:13:39 +03:00 | 125 commits to main since this releaseEven with matching secrets and reachable peer URLs, cross-node metrics can
black-hole: the metric-stream HMAC carries a timestamp and the receiver
rejects anything outside a 5-minute clock-skew window. Two cluster peers
whose clocks drift (common on SBCs / containers without reliable NTP) then
reject every batch in BOTH directions with 'timestamp out of allowed clock
skew', while Raft (no HMAC) and per-node auth keep working — so it looks
healthy but stats never sync.- bridge.VerifyWithSkew(secret, header, ts, body, maxSkew): a maxSkew<=0
skips the freshness check and only verifies the HMAC. The intra-cluster
metric stream is a trusted, authenticated LAN channel where replaying a
node's current metrics is harmless (overwritten next tick), so same-cluster
posts now skip the window; the cross-cluster bridge (untrusted internet)
keeps the strict 5-minute window. - metricstream Sender: log a peer that persistently rejects metric posts
(status + body snippet, rate-limited ~once/30s) so a black-holed stream is
no longer invisible (signature mismatch / clock skew / wrong path all show).
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01U3LvJu9qZfspPr21TW9ptvDownloads
- bridge.VerifyWithSkew(secret, header, ts, body, maxSkew): a maxSkew<=0