Compare commits

...

6 Commits

Author SHA1 Message Date
agent_coder 600cec6ccc fix(docker): make the API-version floor undefeatable (Ping-based) + pin 1.24 + honor caller timeout (#35 review round 1)
Round-1 review found the floor was DEFEATED for the main #34 target (daemon unreachable
at construction): the SDK keeps ClientVersion() == "1.51" on a ping failure (not empty),
so the old empty-string guard passed, the floor branch was skipped, lazy negotiation
stayed armed, and the client could downgrade below 1.24 on the first real request.

- Detect unreachable EXPLICITLY via cli.Ping(ctx) instead of inspecting ClientVersion():
  ping error -> rebuildAtFloor; otherwise NegotiateAPIVersionPing(ping) applies the
  advertised version AND marks the client negotiated (disarms the lazy re-ping); if the
  result is empty or < 1.24 -> rebuildAtFloor. rebuildAtFloor closes and rebuilds with
  WithVersion(minSupportedAPIVersion) — which sets manualOverride, making both the lazy
  checkVersion and NegotiateAPIVersion no-ops, so the pinned client can NEVER re-negotiate
  below the floor. The rebuild reuses the same opts, so agent-signature headers / TLS /
  custom transport are preserved.
- Pin the floor at minSupportedAPIVersion "1.24", not "1.44": 1.44 is "too new" for a
  genuinely old daemon (400) and would make container.go's `< 1.44` MAC-cleanup wrongly
  skip. 1.24 satisfies the invariant and every supported daemon accepts it. Dropped the
  1.44 const.
- resolveNegotiateTimeout(timeout): use the caller's timeout when set (deployer's 3600s,
  autoupdate/autoheal), else a modest 10s default. The proxy hotpath and snapshot loop
  pass nil -> bounded 10s, so a daemon that accepts TCP but hangs on /_ping can't stall
  the whole snapshot loop for a huge fixed time.
- Test TestNegotiateWithFloor_UnreachableThenBelowFloor: unreachable at construction, then
  the daemon advertises 1.20; asserts the client pins to 1.24, ContainerStart goes /v1.24/
  (not /v1.20/), and /_ping isn't hit again after construction (no lazy re-negotiation).
  Mutation-verified: reverting to the old logic reds it.

go build / vet / gofmt clean; go test ./docker/client/ -race 4/4 pass. Still framed as a
hardening (part of #34) — the vendored SDK's ContainerStart sends no body regardless, so
confirm the failing deployment's build before treating this as the complete #34 fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 23:36:14 +03:00
agent_coder dbd2d06659 fix(docker): harden client API-version resolution — eager negotiate + >=1.24 floor (part of #34)
Issue #34 reports container Recreate / native auto-update (#28, #10) failing on modern
Docker Engines via the agent with "starting container with non-empty request body ...
removed in v1.24". The proposed fix: make API-version resolution robust in all three
docker client constructors.

- api/docker/client/client.go: add a shared negotiateWithFloor() used by createLocalClient,
  createTCPClient, createAgentClient. It eagerly NegotiateAPIVersion (10s bound) instead of
  relying on lazy first-call negotiation, and enforces a modern floor: if the negotiated
  version is empty or < 1.24 (versions.LessThan), rebuild pinned at 1.44 (WithVersion, which
  the SDK requires INSTEAD of negotiation — WithVersion sets manualOverride, disabling
  NegotiateAPIVersion). Negotiation stays primary; the floor is only a safety net. Effective
  API version is now guaranteed >= 1.24. Custom transport / TLS / agent headers untouched.
- Tests (api/docker/client/client_test.go): modern daemon -> empty start body; below-floor
  daemon -> pinned to floor; unreachable daemon -> stays >= 1.24.

IMPORTANT FINDING (needs maintainer confirmation before this is called a complete fix for
#34): on the pinned SDK (github.com/docker/docker v28.5.2), ContainerStart posts a nil body
UNCONDITIONALLY (client/container_start.go: cli.post(ctx, ".../start", query, nil, nil) — no
version branch), and NewClientWithOpts defaults to version 1.51 (negotiation only downgrades).
So on develop, the server-side path the issue names (autoupdate.go -> Recreate ->
cli.ContainerStart) cannot emit a request body regardless of API version — this change hardens
the client path but is unlikely to be the actual source of the reported symptom. The
"non-empty body" more likely originates from a DIFFERENT layer (an OLDER docker SDK in the
failing deployment, where ContainerStart did attach HostConfig at API < 1.24; or the docker
PROXY forwarding a raw browser request). Recommend confirming the failing environment's build
before merging as the #34 fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 23:05:26 +03:00
vvzvlad 68592337f5 Merge pull request 'feat(automation): входящий registry-push вебхук для мгновенного автообновления (#32)' (#33) from feat/32-registry-webhook into develop
Reviewed-on: #33
2026-07-05 16:51:48 +03:00
claude code agent 400ac05864 fix(#32): webhook URL via shared helper, disabled-state note, CAS-release test, Conflict helper (review round 1)
F1 [WARNING] The frontend built the trigger URL from window.location.origin,
bypassing the shared webhookHelper whose getBaseUrl() honours the reverse-proxy
sub-path (<base href>) and the desktop file:// build. Under a non-root base-href
deploy the copied URL would miss the sub-path prefix. Added
containerAutomationWebhookUrl(token) to webhookHelper (mirroring dockerWebhookUrl)
and used it; the test asserts against the same helper.

F2 [WARNING] The panel showed the live URL + 'POST runs a pass' text even when
auto-update was disabled — contradicting the server, which 409s a valid token
while disabled. WebhookTriggerSection now takes the enabled flag and shows an
orange note ('trigger is inactive... a POST returns 409') when a token exists but
auto-update is off; the token/Regenerate/Clear stay (the token is kept). Tests
cover both the disabled note and its absence when enabled.

F3 [WARNING] The CAS-release of the REAL runUpdatePass was untested (only the
acquire-FAIL path was). A broken release defer would wedge auto-update forever
(poll drops every tick, webhook worker spins) with a green suite. Added
TestRunUpdatePassReleasesLock: a test store with auto-update enabled and no
endpoints runs a pass to completion and asserts the lock is released (and a
second pass re-acquires it).

F4 [low] Swapped raw NewError(StatusConflict) for the httperror.Conflict helper,
matching the rest of the file and webhook_create.go.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 06:30:38 +03:00
claude code agent b6ba9b46c7 feat(automation): inbound registry-push webhook for immediate auto-update (#32)
Adds POST /api/webhooks/container-automation/{token} which kicks a full
auto-update pass immediately, instead of waiting up to PollInterval (6h) for the
poll. Registry-agnostic 'general kick': the payload is not parsed; the pass
compares digests and updates only stale containers. Polling stays as a fallback.

- Service.TriggerUpdate(): a buffer-1 non-blocking kick channel + a worker
  goroutine on baseCtx that debounces ~10s (collapsing a multi-arch/multi-tag
  push burst into one pass). update() is split into runUpdatePass() bool, which
  returns false when the updateRunning CAS is not acquired; the worker backs off
  and re-runs so a kick landing during a scheduled pass is never lost. The poll
  timer keeps calling the same pass.
- Worker panics are recovered (poll parity — the poll path runs under the
  scheduler's cron.Recover), so a panic in the pass cannot crash the daemon from
  this bare goroutine. runUpdatePass also re-checks AutoUpdate.Enabled as
  defense-in-depth against a disable during the debounce window.
- Route via bouncer.PublicAccess (distinct from /webhooks/{token} by segment
  count); token compared with subtle.ConstantTimeCompare; empty/mismatched token
  -> 404, disabled auto-update -> 409, success -> 202. Token is server-generated
  (uuid) via RegenerateWebhookToken / ClearWebhookToken settings actions; never
  accepted from the client; excluded from /settings/public (present in admin GET
  for the UI).
- Frontend: an AutoUpdatePanel 'Update trigger webhook' section (readonly URL,
  copy, Generate/Regenerate, Clear, registry-setup hint).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 06:05:21 +03:00
vvzvlad 7d71e9e788 Merge pull request 'fix(migrator): heal 2.44.0 stacks with StackFileVersion>0 but missing v{N} dir (#30)' (#31) from fix/30-migration-heal into develop
Reviewed-on: #31
2026-07-03 01:33:36 +03:00
17 changed files with 1248 additions and 18 deletions
+28 -7
View File
@@ -26,20 +26,41 @@ const (
)
// update runs a single auto-update pass over every reachable Docker endpoint.
// It is registered with the scheduler and guarded against overlapping ticks by
// the Service. Errors are logged per endpoint/container so one failure does not
// abort the whole pass; it always returns nil so the scheduler keeps the job.
// It is the scheduler entry point (poll timer): it delegates to runUpdatePass and
// always returns nil so the scheduler keeps the job regardless of the outcome.
func (s *Service) update() error {
s.runUpdatePass()
return nil
}
// runUpdatePass performs a single auto-update pass over every reachable Docker
// endpoint. It is guarded against overlapping runs by the updateRunning CAS: if a
// pass (scheduled or webhook-triggered) is already in progress it returns false
// WITHOUT running, so the caller can decide what to do (the poll timer drops the
// tick; the webhook worker waits and re-runs so a mid-pass kick is not lost).
// Errors are logged per endpoint/container so one failure does not abort the whole
// pass. Returns true when this call actually acquired the lock and ran the pass.
func (s *Service) runUpdatePass() bool {
if !s.updateRunning.CompareAndSwap(false, true) {
log.Debug().Msg("auto-update: previous run still in progress, skipping tick")
return nil
return false
}
defer s.updateRunning.Store(false)
settings, err := s.dataStore.Settings().Settings()
if err != nil {
log.Warn().Err(err).Msg("auto-update: unable to read settings")
return nil
return true
}
// Defense-in-depth: auto-update may have been disabled during the webhook
// debounce/backoff window (the handler's 409 gate and the scheduler removal on
// Reload both act earlier, but neither covers a kick already in flight). Re-read
// the live flag and no-op if it was turned off. Returns true (lock acquired,
// nothing to retry).
if !settings.ContainerAutomation.AutoUpdate.Enabled {
log.Debug().Msg("auto-update: disabled, skipping pass")
return true
}
scope := ScopeLabeled
@@ -56,7 +77,7 @@ func (s *Service) update() error {
endpoints, err := s.dataStore.Endpoint().Endpoints()
if err != nil {
log.Warn().Err(err).Msg("auto-update: unable to list environments")
return nil
return true
}
for i := range endpoints {
@@ -84,7 +105,7 @@ func (s *Service) update() error {
// pruneRetries), so the loop-guard map cannot grow unbounded.
s.pruneRolledBack(time.Now())
return nil
return true
}
// updateOptions carries the per-pass auto-update toggles resolved from settings.
+131 -4
View File
@@ -29,6 +29,16 @@ const (
// It is conservative (hours) to stay within registry rate limits; the image-status cache is
// short-lived (keyed by the local imageID), so each poll re-checks the remote digest.
defaultPollInterval = 6 * time.Hour
// webhookDebounceWindow collapses a burst of inbound registry-push webhooks into a
// single update pass. A multi-arch / multi-tag push fires several deliveries in a
// row; after the first kick the worker waits this long, draining any further kicks,
// so the burst runs one pass instead of one per delivery.
webhookDebounceWindow = 10 * time.Second
// webhookRetryBackoff is how long the webhook worker waits before re-running a pass
// that could not acquire the update lock because a scheduled pass was already in
// progress. It keeps the retry from spinning while ensuring a mid-pass kick is not
// lost until the next poll.
webhookRetryBackoff = 5 * time.Second
)
// Service manages the lifecycle of the auto-heal and auto-update scheduler jobs
@@ -54,6 +64,19 @@ type Service struct {
// default is logNotifier; the field is the seam external senders plug into.
notifier Notifier
// kickCh signals the webhook worker to run an out-of-band update pass. It has a
// buffer of 1 and TriggerUpdate sends non-blockingly, so a burst of concurrent
// triggers naturally coalesces into at most one pending kick.
kickCh chan struct{}
// debounceWindow / retryBackoff are the worker timings, held as fields (defaulted
// from the package constants in NewService) so tests can shrink them.
debounceWindow time.Duration
retryBackoff time.Duration
// updatePass is the seam the webhook worker runs; it defaults to runUpdatePass and
// is overridable in tests to observe the kick/debounce/retry behaviour without a
// live engine. It returns whether the pass acquired the update lock and ran.
updatePass func() bool
mu sync.Mutex
healJobID string
updateJobID string
@@ -98,7 +121,7 @@ func NewService(
baseCtx = context.Background()
}
return &Service{
s := &Service{
baseCtx: baseCtx,
scheduler: scheduler,
dataStore: dataStore,
@@ -109,9 +132,113 @@ func NewService(
// The webhook reads the current settings per-event from the datastore, so a
// URL change in the UI takes effect without a restart; logNotifier keeps the
// existing structured log output unchanged.
notifier: multiNotifier{logNotifier{}, newWebhookNotifier(dataStore)},
retries: make(map[string]retryState),
rolledBack: make(map[string]rolledBackTarget),
notifier: multiNotifier{logNotifier{}, newWebhookNotifier(dataStore)},
kickCh: make(chan struct{}, 1),
debounceWindow: webhookDebounceWindow,
retryBackoff: webhookRetryBackoff,
retries: make(map[string]retryState),
rolledBack: make(map[string]rolledBackTarget),
}
s.updatePass = s.runUpdatePass
// The webhook worker lives for the whole application lifetime (bounded by
// baseCtx), independently of whether the poll job is scheduled: a kick must be
// serviceable even between polls.
go s.triggerWorker()
return s
}
// TriggerUpdate requests an immediate auto-update pass out of band from the poll
// timer. It is called by the inbound registry-push webhook handler so a container
// updates within seconds of a push instead of waiting up to a full poll interval.
// The send is non-blocking onto the buffer-1 kick channel: a burst of concurrent
// triggers coalesces into at most one pending kick, and the worker then debounces
// the burst and, if a scheduled pass is already running, waits and re-runs so a
// kick that lands mid-pass is not lost. Safe to call on a zero-value Service (a
// nil kick channel makes it a no-op).
func (s *Service) TriggerUpdate() {
select {
case s.kickCh <- struct{}{}:
default:
// A kick is already pending (or the channel is nil on a zero-value service):
// the coming pass will cover this trigger too, so drop the duplicate.
}
}
// triggerWorker services webhook-driven update kicks for the life of baseCtx. It
// waits for a kick, debounces the burst, then runs an update pass — re-running
// after a short backoff if a scheduled pass currently holds the update lock, so a
// kick that arrives mid-pass is not dropped until the next poll.
func (s *Service) triggerWorker() {
for {
// Block until the first kick of a burst (or shutdown).
select {
case <-s.baseCtx.Done():
return
case <-s.kickCh:
}
if !s.debounceKicks() {
return // shutdown during the debounce window
}
// Run the pass, retrying while a scheduled pass holds the lock. runUpdatePass
// returns false only when the CAS is not acquired (another pass in progress);
// backing off and retrying guarantees this kick eventually runs a fresh pass
// that observes the just-pushed image, rather than being silently dropped.
//
// runPassRecovered wraps each attempt in a recover(): the poll path runs under
// the scheduler's cron.Recover, so a panic there is logged and the daemon
// survives. This webhook worker is a bare goroutine, so without its own
// recover an unrelated panic in the update pass (reachable by a token holder)
// would crash the whole process. Match the poll path's guarantee.
for !s.runPassRecovered() {
log.Debug().Msg("auto-update: webhook-triggered pass deferred, a scheduled pass is running; will retry")
select {
case <-s.baseCtx.Done():
return
case <-time.After(s.retryBackoff):
}
}
}
}
// runPassRecovered runs one update pass with a recover() so a panic in the pass
// cannot crash the process from this bare worker goroutine (the poll path is
// already protected by the scheduler's cron.Recover). It returns whatever the
// pass returned; on a recovered panic it returns true ("done") so the retry loop
// does not spin on a deterministically-panicking pass — the next kick starts
// fresh. runUpdatePass releases the updateRunning CAS via defer even on panic, so
// no lock is leaked here.
func (s *Service) runPassRecovered() (done bool) {
defer func() {
if r := recover(); r != nil {
log.Error().Interface("panic", r).Msg("auto-update: recovered from panic in webhook-triggered update pass")
done = true
}
}()
return s.updatePass()
}
// debounceKicks waits a fixed window after the first kick, draining any further
// kicks that arrive during it, so a burst of registry deliveries collapses into a
// single pass. It returns false if baseCtx is cancelled while waiting.
func (s *Service) debounceKicks() bool {
timer := time.NewTimer(s.debounceWindow)
defer timer.Stop()
for {
select {
case <-s.baseCtx.Done():
return false
case <-s.kickCh:
// Additional kick within the window: coalesce it (fixed window from the
// first kick, not reset — a bounded, single collapsed pass).
case <-timer.C:
return true
}
}
}
+207
View File
@@ -0,0 +1,207 @@
package containerautomation
import (
"context"
"sync"
"testing"
"time"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/require"
)
// newTriggerTestService builds a Service wired only for the webhook trigger worker,
// with a stubbed updatePass seam and short timings, and starts the worker on a
// cancellable context that is torn down at the end of the test.
func newTriggerTestService(t *testing.T, pass func() bool) *Service {
t.Helper()
ctx, cancel := context.WithCancel(context.Background())
t.Cleanup(cancel)
s := &Service{
baseCtx: ctx,
kickCh: make(chan struct{}, 1),
debounceWindow: 40 * time.Millisecond,
retryBackoff: 10 * time.Millisecond,
updatePass: pass,
}
go s.triggerWorker()
return s
}
// TestTriggerUpdateCoalescesBurst verifies that a burst of TriggerUpdate calls
// arriving within the debounce window collapses into a single update pass, so a
// multi-arch / multi-tag registry push does not run one pass per delivery.
func TestTriggerUpdateCoalescesBurst(t *testing.T) {
var mu sync.Mutex
passes := 0
s := newTriggerTestService(t, func() bool {
mu.Lock()
passes++
mu.Unlock()
return true
})
// A burst of deliveries in a tight loop, all well within the debounce window.
for range 10 {
s.TriggerUpdate()
}
// Wait out the debounce window (plus margin) and confirm exactly one pass ran.
require.Eventually(t, func() bool {
mu.Lock()
defer mu.Unlock()
return passes == 1
}, time.Second, 5*time.Millisecond, "the burst should trigger exactly one pass")
// Give any erroneously-scheduled extra pass a chance to appear, then re-assert.
time.Sleep(120 * time.Millisecond)
mu.Lock()
defer mu.Unlock()
require.Equal(t, 1, passes, "no extra pass should run after the coalesced burst")
}
// TestTriggerUpdateRetriesWhenPassBusy verifies the critical edge case: when a
// scheduled pass is already running (runUpdatePass returns false because the
// updateRunning CAS is not acquired), the kick is NOT lost — the worker waits and
// re-runs the pass after the running one finishes.
func TestTriggerUpdateRetriesWhenPassBusy(t *testing.T) {
var mu sync.Mutex
var attempts int
var ranAfterBusy bool
s := newTriggerTestService(t, func() bool {
mu.Lock()
attempts++
n := attempts
if n > 1 {
ranAfterBusy = true
}
mu.Unlock()
// First attempt models a scheduled pass holding the lock: report the CAS
// miss so the worker must back off and retry.
return n > 1
})
s.TriggerUpdate()
require.Eventually(t, func() bool {
mu.Lock()
defer mu.Unlock()
return ranAfterBusy && attempts >= 2
}, time.Second, 5*time.Millisecond, "the worker must retry and run a fresh pass after the CAS miss")
}
// TestTriggerUpdateBufferedDuringRunningPass verifies that a kick arriving WHILE a
// pass is already running is not lost: the buffer-1 channel holds it, and the
// worker reads it on the next loop iteration and runs a second pass. This is the
// coalescing path for a push that lands mid-pass (distinct from the CAS-miss retry
// path, which models a SCHEDULED pass holding the lock).
func TestTriggerUpdateBufferedDuringRunningPass(t *testing.T) {
var mu sync.Mutex
var passes int
release := make(chan struct{})
firstStarted := make(chan struct{})
s := newTriggerTestService(t, func() bool {
mu.Lock()
passes++
n := passes
mu.Unlock()
if n == 1 {
close(firstStarted) // signal the first pass is in progress
<-release // ...and block here until the test releases it
}
return true
})
s.TriggerUpdate()
<-firstStarted // the first pass is now running
// A second push lands while the first pass is still running. It must be
// buffered (not dropped) and processed after the current pass finishes.
s.TriggerUpdate()
close(release)
require.Eventually(t, func() bool {
mu.Lock()
defer mu.Unlock()
return passes == 2
}, time.Second, 5*time.Millisecond, "a kick during a running pass must run a second pass, not be lost")
}
// TestTriggerWorkerRecoversFromPanic verifies the webhook worker survives a panic in
// the update pass (poll parity: the poll path is protected by cron.Recover). Without
// the recover, a panic in this bare goroutine would crash the whole test binary — so
// the fact that a LATER kick still runs a pass proves recovery works.
func TestTriggerWorkerRecoversFromPanic(t *testing.T) {
var mu sync.Mutex
var attempts int
s := newTriggerTestService(t, func() bool {
mu.Lock()
attempts++
n := attempts
mu.Unlock()
if n == 1 {
panic("boom in the update pass")
}
return true
})
s.TriggerUpdate() // first pass panics; the worker must survive
require.Eventually(t, func() bool {
mu.Lock()
defer mu.Unlock()
return attempts == 1
}, time.Second, 5*time.Millisecond, "the panicking pass must have been attempted")
// The worker is still alive: a second kick runs a clean pass.
s.TriggerUpdate()
require.Eventually(t, func() bool {
mu.Lock()
defer mu.Unlock()
return attempts >= 2
}, time.Second, 5*time.Millisecond, "the worker must survive the panic and run later kicks")
}
// TestRunUpdatePassCASGuard verifies that runUpdatePass reports whether it acquired
// the overlap lock: a call while a pass is already in progress returns false without
// running, which is what lets the poll timer drop overlapping ticks and the webhook
// worker know it must retry rather than silently losing the kick.
func TestRunUpdatePassCASGuard(t *testing.T) {
s := &Service{baseCtx: context.Background()}
// Simulate an in-progress pass by holding the CAS as runUpdatePass would.
require.True(t, s.updateRunning.CompareAndSwap(false, true))
require.False(t, s.runUpdatePass(), "runUpdatePass must return false when a pass is already running")
}
// TestRunUpdatePassReleasesLock pins the CAS-RELEASE contract of the real
// runUpdatePass (not the stub): after a pass completes, the updateRunning lock
// MUST be released, or every future pass — poll tick AND webhook kick — would be
// permanently wedged (the poll drops the tick, the webhook worker spins its retry
// loop forever) while the rest of the suite stays green. A test store with
// auto-update enabled and no endpoints runs a trivial pass to completion.
func TestRunUpdatePassReleasesLock(t *testing.T) {
_, store := datastore.MustNewTestStore(t, true, false)
settings, err := store.Settings().Settings()
require.NoError(t, err)
settings.ContainerAutomation.AutoUpdate.Enabled = true
require.NoError(t, store.Settings().UpdateSettings(settings))
s := &Service{baseCtx: context.Background(), dataStore: store}
// First pass: acquires the lock and runs to completion (no endpoints → no work).
require.True(t, s.runUpdatePass(), "the pass should acquire the lock and run")
require.False(t, s.updateRunning.Load(), "the lock MUST be released after the pass")
// A second pass must be able to acquire the lock again — proves the release,
// not just the flag read.
require.True(t, s.runUpdatePass(), "a subsequent pass must re-acquire the released lock")
require.False(t, s.updateRunning.Load())
}
+101 -4
View File
@@ -2,6 +2,7 @@ package client
import (
"bytes"
"context"
"errors"
"fmt"
"io"
@@ -16,6 +17,7 @@ import (
"github.com/rs/zerolog/log"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/client"
"github.com/segmentio/encoding/json"
)
@@ -25,6 +27,24 @@ var errUnsupportedEnvironmentType = errors.New("environment not supported")
const (
defaultDockerRequestTimeout = 60 * time.Second
dockerClientVersion = "1.37"
// defaultNegotiateTimeout bounds eager API-version negotiation at client
// construction when the caller does not supply an explicit timeout, so an
// unreachable or hung daemon cannot block construction for long. Callers
// that pass a timeout (e.g. remote stack deploys, container automation)
// override this with their own deliberate value.
defaultNegotiateTimeout = 10 * time.Second
// minSupportedAPIVersion is both the hard invariant every Docker client
// must satisfy and the version the client is pinned to when negotiation
// cannot determine a usable daemon version. Below API v1.24 the daemon
// rejects requests the SDK still shapes with legacy semantics (e.g. a
// non-empty body on "POST /containers/{id}/start"), which breaks container
// recreate (issue #34). 1.24 is accepted by every supported daemon, so it
// is safe to pin even against a genuinely old engine (a newer floor such as
// 1.44 would be rejected as "too new" and would also skip the
// pre-1.44 MAC-address cleanup in container.go).
minSupportedAPIVersion = "1.24"
)
type NodeNamesCtxKey struct{}
@@ -71,11 +91,88 @@ func (factory *ClientFactory) CreateClient(endpoint *portainer.Endpoint, nodeNam
return createTCPClient(endpoint, timeout)
}
// resolveNegotiateTimeout derives the eager-negotiation timeout from the
// caller's requested client timeout, respecting deliberate per-endpoint
// timeouts (e.g. remote stack deploys, container automation) and falling back
// to a modest default when none is provided.
func resolveNegotiateTimeout(timeout *time.Duration) time.Duration {
if timeout != nil {
return *timeout
}
return defaultNegotiateTimeout
}
// negotiateWithFloor builds a Docker client from opts (which must include
// client.WithAPIVersionNegotiation()) and resolves its effective API version at
// construction time, guaranteeing it never drops below minSupportedAPIVersion —
// not even lazily on a later request. The custom agent/TCP transports do not
// reliably resolve a modern API version through the SDK's lazy first-call
// negotiation, so we resolve it up-front here.
//
// We ping the daemon explicitly (bounded by timeout) rather than relying on
// cli.NegotiateAPIVersion + inspecting ClientVersion(), because that cannot
// tell "reached the daemon and negotiated" apart from "never reached it": the
// SDK seeds every client with api.DefaultVersion and NegotiateAPIVersion
// swallows a ping error without clearing the version or marking the client as
// negotiated (docker@v28.5.2 client.go NegotiateAPIVersion). A non-empty
// ClientVersion() would therefore let an unreachable daemon pass, leaving lazy
// negotiation armed to downgrade below the floor on the first real request.
//
// - Ping error (unreachable at construction): pin at the floor.
// - Ping ok but advertised version < floor (or empty): pin at the floor.
// - Ping ok and version >= floor: apply it via NegotiateAPIVersionPing, which
// also marks the client negotiated so it will not re-ping lazily.
//
// The floor is applied by rebuilding with client.WithVersion, which sets
// manualOverride=true and thereby disables all further (lazy) negotiation, so
// the effective version is fixed and can never re-negotiate below the floor.
func negotiateWithFloor(opts []client.Opt, timeout time.Duration) (*client.Client, error) {
cli, err := client.NewClientWithOpts(opts...)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
ping, err := cli.Ping(ctx)
if err != nil {
// Daemon unreachable at construction: pin at the floor so the client
// cannot lazily negotiate below it on the first real request.
return rebuildAtFloor(cli, opts)
}
// Apply the version advertised by the daemon; this also disarms lazy
// re-negotiation on subsequent requests.
cli.NegotiateAPIVersionPing(ping)
if v := cli.ClientVersion(); v == "" || versions.LessThan(v, minSupportedAPIVersion) {
return rebuildAtFloor(cli, opts)
}
return cli, nil
}
// rebuildAtFloor closes cli and returns a new client from the same opts pinned
// at minSupportedAPIVersion. client.WithVersion sets manualOverride=true, which
// makes both the lazy checkVersion and NegotiateAPIVersion no-ops, so the
// effective API version stays fixed at the floor.
func rebuildAtFloor(cli *client.Client, opts []client.Opt) (*client.Client, error) {
if err := cli.Close(); err != nil {
log.Warn().Err(err).Msg("failed to close docker client before applying API version floor")
}
return client.NewClientWithOpts(append(opts, client.WithVersion(minSupportedAPIVersion))...)
}
func createLocalClient(endpoint *portainer.Endpoint) (*client.Client, error) {
return client.NewClientWithOpts(
opts := []client.Opt{
client.WithHost(endpoint.URL),
client.WithAPIVersionNegotiation(),
)
}
return negotiateWithFloor(opts, defaultNegotiateTimeout)
}
func createTCPClient(endpoint *portainer.Endpoint, timeout *time.Duration) (*client.Client, error) {
@@ -94,7 +191,7 @@ func createTCPClient(endpoint *portainer.Endpoint, timeout *time.Duration) (*cli
opts = append(opts, client.WithScheme("https"))
}
return client.NewClientWithOpts(opts...)
return negotiateWithFloor(opts, resolveNegotiateTimeout(timeout))
}
func createAgentClient(endpoint *portainer.Endpoint, endpointURL string, signatureService portainer.DigitalSignatureService, nodeName string, timeout *time.Duration) (*client.Client, error) {
@@ -128,7 +225,7 @@ func createAgentClient(endpoint *portainer.Endpoint, endpointURL string, signatu
opts = append(opts, client.WithScheme("https"))
}
return client.NewClientWithOpts(opts...)
return negotiateWithFloor(opts, resolveNegotiateTimeout(timeout))
}
type NodeNameTransport struct {
+171
View File
@@ -1,11 +1,20 @@
package client
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/pkg/fips"
dockercontainer "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/client"
"github.com/stretchr/testify/require"
)
@@ -29,3 +38,165 @@ func TestHttpClient(t *testing.T) {
require.Error(t, err)
require.Nil(t, cli)
}
// startRecord captures the request made to POST /containers/{id}/start.
type startRecord struct {
path string
body []byte
}
// newDockerStub returns an httptest server that mimics a Docker daemon
// advertising apiVersion, and records the request made to the container start
// endpoint into rec.
func newDockerStub(t *testing.T, apiVersion string, rec *startRecord) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.HasSuffix(r.URL.Path, "/_ping"):
w.Header().Set("Api-Version", apiVersion)
w.Header().Set("Ostype", "linux")
w.WriteHeader(http.StatusOK)
case strings.HasSuffix(r.URL.Path, "/start"):
body, _ := io.ReadAll(r.Body)
rec.path = r.URL.Path
rec.body = body
w.WriteHeader(http.StatusNoContent)
default:
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("{}"))
}
}))
t.Cleanup(srv.Close)
return srv
}
// TestNegotiateWithFloor_ModernDaemon verifies that against a daemon
// advertising a modern API version (>= 1.24) the client negotiates a modern
// version and issues POST /containers/{id}/start with an EMPTY body — the
// behavior required by API >= 1.24 (regression test for issue #34).
func TestNegotiateWithFloor_ModernDaemon(t *testing.T) {
t.Parallel()
var rec startRecord
srv := newDockerStub(t, "1.54", &rec)
opts := []client.Opt{
client.WithHost(srv.URL),
client.WithAPIVersionNegotiation(),
}
cli, err := negotiateWithFloor(opts, defaultNegotiateTimeout)
require.NoError(t, err)
t.Cleanup(func() { _ = cli.Close() })
// Negotiation resolved a concrete, modern version (capped at the client's
// maximum supported version).
require.NotEmpty(t, cli.ClientVersion())
require.False(t, versions.LessThan(cli.ClientVersion(), minSupportedAPIVersion),
"effective API version %q must be >= %q", cli.ClientVersion(), minSupportedAPIVersion)
err = cli.ContainerStart(context.Background(), "abc123", dockercontainer.StartOptions{})
require.NoError(t, err)
// The request must be versioned and carry no HostConfig body.
require.Contains(t, rec.path, "/v"+cli.ClientVersion()+"/")
require.Empty(t, rec.body, "start request body must be empty on API >= 1.24")
}
// TestNegotiateWithFloor_BelowFloorDaemon verifies the floor branch: a daemon
// that advertises an API version below the supported floor causes negotiation
// to resolve a legacy version, and the client must then be pinned at
// minSupportedAPIVersion instead of issuing requests below the supported floor.
func TestNegotiateWithFloor_BelowFloorDaemon(t *testing.T) {
t.Parallel()
var rec startRecord
// 1.20 is below minSupportedAPIVersion (1.24); negotiation would downgrade
// the client to it, which is exactly what must NOT be used for requests.
srv := newDockerStub(t, "1.20", &rec)
opts := []client.Opt{
client.WithHost(srv.URL),
client.WithAPIVersionNegotiation(),
}
cli, err := negotiateWithFloor(opts, defaultNegotiateTimeout)
require.NoError(t, err)
t.Cleanup(func() { _ = cli.Close() })
// The client is pinned at the supported floor, not the legacy 1.20.
require.Equal(t, minSupportedAPIVersion, cli.ClientVersion())
}
// TestNegotiateWithFloor_UnreachableThenBelowFloor covers the exact issue #34
// scenario the reviewer reproduced: the daemon is UNREACHABLE at construction
// time, then comes back up advertising a legacy version (1.20). The client must
// pin at the floor during construction AND must not lazily re-negotiate down to
// the legacy version on the first real request.
func TestNegotiateWithFloor_UnreachableThenBelowFloor(t *testing.T) {
t.Parallel()
var (
reachable atomic.Bool // false during construction, true afterwards
pingHits atomic.Int32
rec startRecord
)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.HasSuffix(r.URL.Path, "/_ping"):
pingHits.Add(1)
if !reachable.Load() {
// Unreachable: make Ping fail (both HEAD and its GET fallback).
w.WriteHeader(http.StatusServiceUnavailable)
return
}
// Daemon is now up, but advertises a legacy, below-floor version.
w.Header().Set("Api-Version", "1.20")
w.Header().Set("Ostype", "linux")
w.WriteHeader(http.StatusOK)
case strings.HasSuffix(r.URL.Path, "/start"):
body, _ := io.ReadAll(r.Body)
rec.path = r.URL.Path
rec.body = body
w.WriteHeader(http.StatusNoContent)
default:
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("{}"))
}
}))
t.Cleanup(srv.Close)
opts := []client.Opt{
client.WithHost(srv.URL),
client.WithAPIVersionNegotiation(),
}
// Construction happens while the daemon is unreachable.
cli, err := negotiateWithFloor(opts, defaultNegotiateTimeout)
require.NoError(t, err)
t.Cleanup(func() { _ = cli.Close() })
require.Equal(t, minSupportedAPIVersion, cli.ClientVersion(),
"unreachable daemon must pin the client at the floor")
pingsAfterConstruction := pingHits.Load()
// The daemon comes back up advertising a legacy version.
reachable.Store(true)
err = cli.ContainerStart(context.Background(), "abc123", dockercontainer.StartOptions{})
require.NoError(t, err)
// The floor must hold: the request is issued at the pinned version, not the
// daemon's legacy 1.20, and the client did not re-ping to re-negotiate.
require.Contains(t, rec.path, "/v"+minSupportedAPIVersion+"/",
"start must be issued at the pinned floor version, not the legacy 1.20")
require.NotContains(t, rec.path, "/v1.20/")
require.Equal(t, pingsAfterConstruction, pingHits.Load(),
"client must not lazily re-negotiate after being pinned at the floor")
}
@@ -18,6 +18,7 @@ import (
"github.com/portainer/portainer/pkg/libhttp/ssrf"
"github.com/portainer/portainer/pkg/validate"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/rs/zerolog/log"
"golang.org/x/oauth2"
@@ -104,6 +105,14 @@ type autoUpdateSettingsPayload struct {
Cleanup *bool `example:"false"`
RollbackOnFailure *bool `example:"false"`
RollbackTimeout *string `example:"120s"`
// RegenerateWebhookToken, when true, (re)generates the inbound registry-push
// webhook token server-side. The client never supplies the token value itself;
// it only requests this action, so the secret is always a fresh server uuid.
RegenerateWebhookToken *bool `example:"false"`
// ClearWebhookToken, when true, removes the webhook token and thereby disables
// the inbound trigger endpoint. RegenerateWebhookToken takes precedence if both
// are set.
ClearWebhookToken *bool `example:"false"`
}
func (payload *settingsUpdatePayload) Validate(r *http.Request) error {
@@ -373,6 +382,21 @@ func (handler *Handler) updateSettings(tx dataservices.DataStoreTx, payload sett
current.Cleanup = *cmp.Or(autoUpdate.Cleanup, &current.Cleanup)
current.RollbackOnFailure = *cmp.Or(autoUpdate.RollbackOnFailure, &current.RollbackOnFailure)
current.RollbackTimeout = *cmp.Or(autoUpdate.RollbackTimeout, &current.RollbackTimeout)
// Webhook token actions. The token value is never accepted from the client:
// regenerate mints a fresh server-side uuid (enabling/rotating the inbound
// trigger endpoint), clear removes it (disabling the endpoint). Regenerate wins
// if both are set.
switch {
case autoUpdate.RegenerateWebhookToken != nil && *autoUpdate.RegenerateWebhookToken:
token, err := uuid.NewRandom()
if err != nil {
return nil, httperror.InternalServerError("Unable to generate a webhook token", err)
}
current.WebhookToken = token.String()
case autoUpdate.ClearWebhookToken != nil && *autoUpdate.ClearWebhookToken:
current.WebhookToken = ""
}
}
if payload.ContainerAutomation != nil && payload.ContainerAutomation.Notification != nil {
@@ -0,0 +1,120 @@
package settings
import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/filesystem"
"github.com/google/uuid"
"github.com/stretchr/testify/require"
)
func boolptr(b bool) *bool { return &b }
// newWebhookTokenTestHandler builds a settings Handler backed by a fresh test
// store and a real (temp-dir) file service, enough to exercise updateSettings.
func newWebhookTokenTestHandler(t *testing.T) (*Handler, *datastore.Store) {
t.Helper()
_, store := datastore.MustNewTestStore(t, true, false)
fileService, err := filesystem.NewService(t.TempDir(), "")
require.NoError(t, err)
return &Handler{DataStore: store, FileService: fileService}, store
}
// applyAutoUpdate runs updateSettings inside a transaction with the given
// auto-update payload and returns the resulting settings.
func applyAutoUpdate(t *testing.T, handler *Handler, store *datastore.Store, autoUpdate *autoUpdateSettingsPayload) *portainer.Settings {
t.Helper()
var settings *portainer.Settings
require.NoError(t, store.UpdateTx(func(tx dataservices.DataStoreTx) error {
var e error
settings, e = handler.updateSettings(tx, settingsUpdatePayload{
ContainerAutomation: &containerAutomationSettingsPayload{AutoUpdate: autoUpdate},
})
return e
}))
return settings
}
// TestUpdateSettingsWebhookTokenRegenerate verifies that RegenerateWebhookToken
// mints a fresh server-side uuid, that regenerating again rotates it to a new
// value, and that the client-supplied token value is never trusted (there is no
// field to supply one).
func TestUpdateSettingsWebhookTokenRegenerate(t *testing.T) {
handler, store := newWebhookTokenTestHandler(t)
settings := applyAutoUpdate(t, handler, store, &autoUpdateSettingsPayload{
RegenerateWebhookToken: boolptr(true),
})
first := settings.ContainerAutomation.AutoUpdate.WebhookToken
require.NotEmpty(t, first, "regenerate must set a token")
_, err := uuid.Parse(first)
require.NoError(t, err, "the token must be a valid uuid")
// Persisted to the store.
persisted, err := store.Settings().Settings()
require.NoError(t, err)
require.Equal(t, first, persisted.ContainerAutomation.AutoUpdate.WebhookToken)
// Regenerating again rotates the token.
settings = applyAutoUpdate(t, handler, store, &autoUpdateSettingsPayload{
RegenerateWebhookToken: boolptr(true),
})
second := settings.ContainerAutomation.AutoUpdate.WebhookToken
require.NotEmpty(t, second)
require.NotEqual(t, first, second, "regenerate must rotate the token")
}
// TestUpdateSettingsWebhookTokenClear verifies that ClearWebhookToken removes the
// token (disabling the inbound endpoint).
func TestUpdateSettingsWebhookTokenClear(t *testing.T) {
handler, store := newWebhookTokenTestHandler(t)
settings := applyAutoUpdate(t, handler, store, &autoUpdateSettingsPayload{
RegenerateWebhookToken: boolptr(true),
})
require.NotEmpty(t, settings.ContainerAutomation.AutoUpdate.WebhookToken)
settings = applyAutoUpdate(t, handler, store, &autoUpdateSettingsPayload{
ClearWebhookToken: boolptr(true),
})
require.Empty(t, settings.ContainerAutomation.AutoUpdate.WebhookToken, "clear must remove the token")
}
// TestUpdateSettingsWebhookTokenUntouched verifies that an auto-update save that
// does not request a token action leaves the existing token unchanged (so saving
// other auto-update fields does not accidentally rotate or drop the endpoint).
func TestUpdateSettingsWebhookTokenUntouched(t *testing.T) {
handler, store := newWebhookTokenTestHandler(t)
settings := applyAutoUpdate(t, handler, store, &autoUpdateSettingsPayload{
RegenerateWebhookToken: boolptr(true),
})
token := settings.ContainerAutomation.AutoUpdate.WebhookToken
require.NotEmpty(t, token)
settings = applyAutoUpdate(t, handler, store, &autoUpdateSettingsPayload{
Enabled: boolptr(true),
})
require.Equal(t, token, settings.ContainerAutomation.AutoUpdate.WebhookToken, "a non-action save must not touch the token")
}
// TestUpdateSettingsWebhookTokenRegenerateWins verifies that when both actions are
// set, regenerate takes precedence over clear.
func TestUpdateSettingsWebhookTokenRegenerateWins(t *testing.T) {
handler, store := newWebhookTokenTestHandler(t)
settings := applyAutoUpdate(t, handler, store, &autoUpdateSettingsPayload{
RegenerateWebhookToken: boolptr(true),
ClearWebhookToken: boolptr(true),
})
require.NotEmpty(t, settings.ContainerAutomation.AutoUpdate.WebhookToken, "regenerate must win over clear")
}
+18
View File
@@ -11,12 +11,25 @@ import (
"github.com/gorilla/mux"
)
// ContainerAutomationTrigger triggers an immediate container auto-update pass. It
// is a minimal interface (mirroring the settings handler's
// ContainerAutomationReloader) so the webhooks handler does not depend on the
// concrete container-automation service. It is satisfied by
// *containerautomation.Service.
type ContainerAutomationTrigger interface {
TriggerUpdate()
}
// Handler is the HTTP handler used to handle webhook operations.
type Handler struct {
*mux.Router
requestBouncer security.BouncerService
DataStore dataservices.DataStore
DockerClientFactory *dockerclient.ClientFactory
// ContainerAutomationService receives the inbound registry-push kick that
// triggers an immediate auto-update pass. It may be nil (the endpoint then
// reports the feature as unavailable rather than panicking).
ContainerAutomationService ContainerAutomationTrigger
}
// NewHandler creates a handler to manage webhooks operations.
@@ -35,6 +48,11 @@ func NewHandler(bouncer security.BouncerService) *Handler {
bouncer.AuthenticatedAccess(httperror.LoggerHandler(h.webhookDelete))).Methods(http.MethodDelete)
h.Handle("/webhooks/{token}",
bouncer.PublicAccess(httperror.LoggerHandler(h.webhookExecute))).Methods(http.MethodPost)
// Inbound registry-push trigger for native container auto-update. The extra path
// segment ("container-automation") keeps it distinct from the resource webhook
// above, which matches a single {token} segment.
h.Handle("/webhooks/container-automation/{token}",
bouncer.PublicAccess(httperror.LoggerHandler(h.webhookContainerAutomation))).Methods(http.MethodPost)
return h
}
@@ -0,0 +1,65 @@
package webhooks
import (
"crypto/subtle"
"errors"
"net/http"
httperror "github.com/portainer/portainer/pkg/libhttp/error"
"github.com/portainer/portainer/pkg/libhttp/request"
)
// @summary Trigger a container auto-update pass
// @description Inbound registry-push webhook: any POST with the configured token
// @description immediately runs a full native container auto-update pass (the same
// @description pass the poll timer runs), so a container updates within seconds of a
// @description registry push instead of waiting for the next poll. The payload is not
// @description parsed — the pass itself compares digests and updates only what is
// @description stale, so it is registry-agnostic (Gitea, GHCR, Docker Hub, ...).
// @description **Access policy**: public (guarded by the secret token)
// @tags webhooks
// @param token path string true "Auto-update webhook token"
// @success 202 "Update pass triggered"
// @failure 404 "Unknown or empty token"
// @failure 409 "Container auto-update is disabled"
// @failure 500 "Server error"
// @router /webhooks/container-automation/{token} [post]
func (handler *Handler) webhookContainerAutomation(w http.ResponseWriter, r *http.Request) *httperror.HandlerError {
// An empty/absent token is reported as such via the error; treat it as an empty
// token so it falls through to the constant-time compare below and yields a 404
// (never a 500), matching how a wrong token is handled.
token, _ := request.RetrieveRouteVariableValue(r, "token")
settings, err := handler.DataStore.Settings().Settings()
if err != nil {
return httperror.InternalServerError("Unable to retrieve the settings from the database", err)
}
autoUpdate := settings.ContainerAutomation.AutoUpdate
// Compare in constant time and treat an empty configured token as "endpoint
// disabled": without this an empty request token would match an empty configured
// token and let anyone trigger a pass. A mismatch (or empty token) is a 404 so the
// endpoint does not reveal whether a token is configured.
expected := autoUpdate.WebhookToken
if expected == "" || subtle.ConstantTimeCompare([]byte(token), []byte(expected)) != 1 {
return httperror.NotFound("Invalid webhook token", errors.New("invalid container-automation webhook token"))
}
// The token is valid but auto-update is turned off: report a conflict rather than
// silently accepting a kick that would never do anything.
if !autoUpdate.Enabled {
return httperror.Conflict("Container auto-update is disabled", errors.New("container auto-update is disabled"))
}
if handler.ContainerAutomationService == nil {
return httperror.InternalServerError("Container automation service is not available", errors.New("container automation service not wired"))
}
// Fire-and-forget: the service coalesces/debounces bursts and runs the pass
// asynchronously, so respond 202 Accepted immediately with no body.
handler.ContainerAutomationService.TriggerUpdate()
w.WriteHeader(http.StatusAccepted)
return nil
}
@@ -0,0 +1,104 @@
package webhooks
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/portainer/portainer/api/datastore"
"github.com/gorilla/mux"
"github.com/stretchr/testify/require"
)
// fakeTrigger records how many times the inbound webhook asked for an update pass.
type fakeTrigger struct{ calls int }
func (f *fakeTrigger) TriggerUpdate() { f.calls++ }
// setupWebhookAutomationHandler builds a webhooks.Handler backed by a test store
// whose auto-update settings carry the given token and enabled flag, plus a fake
// trigger to observe the kick.
func setupWebhookAutomationHandler(t *testing.T, token string, enabled bool) (*Handler, *fakeTrigger) {
t.Helper()
_, store := datastore.MustNewTestStore(t, true, false)
settings, err := store.Settings().Settings()
require.NoError(t, err)
settings.ContainerAutomation.AutoUpdate.WebhookToken = token
settings.ContainerAutomation.AutoUpdate.Enabled = enabled
require.NoError(t, store.Settings().UpdateSettings(settings))
trigger := &fakeTrigger{}
handler := &Handler{DataStore: store, ContainerAutomationService: trigger}
return handler, trigger
}
// callWebhook invokes the handler with the given request-path token wired as the
// mux route variable, mirroring the router configuration.
func callWebhook(handler *Handler, requestToken string) (*httptest.ResponseRecorder, int) {
req := httptest.NewRequest(http.MethodPost, "/webhooks/container-automation/"+requestToken, nil)
req = mux.SetURLVars(req, map[string]string{"token": requestToken})
rr := httptest.NewRecorder()
herr := handler.webhookContainerAutomation(rr, req)
if herr != nil {
return rr, herr.StatusCode
}
return rr, rr.Code
}
func TestWebhookContainerAutomation(t *testing.T) {
const token = "11111111-2222-3333-4444-555555555555"
t.Run("valid token on enabled auto-update triggers a pass and returns 202", func(t *testing.T) {
handler, trigger := setupWebhookAutomationHandler(t, token, true)
rr, status := callWebhook(handler, token)
require.Equal(t, http.StatusAccepted, status)
require.Equal(t, 1, trigger.calls, "a valid kick must trigger exactly one pass")
require.Empty(t, rr.Body.String(), "202 response has no body")
})
t.Run("wrong token returns 404 and does not trigger", func(t *testing.T) {
handler, trigger := setupWebhookAutomationHandler(t, token, true)
_, status := callWebhook(handler, "wrong-token")
require.Equal(t, http.StatusNotFound, status)
require.Zero(t, trigger.calls)
})
t.Run("empty request token returns 404", func(t *testing.T) {
handler, trigger := setupWebhookAutomationHandler(t, token, true)
_, status := callWebhook(handler, "")
require.Equal(t, http.StatusNotFound, status)
require.Zero(t, trigger.calls)
})
t.Run("empty configured token cannot be matched (endpoint disabled)", func(t *testing.T) {
// No token configured: even an empty request token must not match, or anyone
// could trigger a pass.
handler, trigger := setupWebhookAutomationHandler(t, "", true)
_, status := callWebhook(handler, "")
require.Equal(t, http.StatusNotFound, status)
require.Zero(t, trigger.calls)
})
t.Run("valid token but auto-update disabled returns 409", func(t *testing.T) {
handler, trigger := setupWebhookAutomationHandler(t, token, false)
_, status := callWebhook(handler, token)
require.Equal(t, http.StatusConflict, status)
require.Zero(t, trigger.calls, "a disabled endpoint must not trigger a pass")
})
}
+1
View File
@@ -299,6 +299,7 @@ func (server *Server) Start(ctx context.Context) error {
var webhookHandler = webhooks.NewHandler(requestBouncer)
webhookHandler.DataStore = server.DataStore
webhookHandler.DockerClientFactory = server.DockerClientFactory
webhookHandler.ContainerAutomationService = server.ContainerAutomationService
server.Handler = &handler.Handler{
RoleHandler: roleHandler,
+9
View File
@@ -1187,6 +1187,15 @@ type (
// recreated back on the previous image. Standalone-only (M5).
RollbackOnFailure bool `json:"RollbackOnFailure"`
RollbackTimeout string `json:"RollbackTimeout" example:"120s"`
// WebhookToken is the secret path segment of the inbound registry-push
// webhook (POST /api/webhooks/container-automation/{token}). A push to the
// registry can call that endpoint to trigger an immediate auto-update pass
// instead of waiting for the next poll. It is SERVER-generated only (a uuid,
// never accepted from the client); empty disables the endpoint. It is
// returned in the admin GET /settings (the UI renders the URL from it) but is
// deliberately excluded from GET /settings/public so it never leaks to
// non-admins.
WebhookToken string `json:"WebhookToken"`
}
// ContainerAutomationNotificationSettings holds the webhook notification
+4
View File
@@ -14,6 +14,10 @@ export function dockerWebhookUrl(token: string) {
return `${baseUrl}${API_ENDPOINT_WEBHOOKS}/${token}`;
}
export function containerAutomationWebhookUrl(token: string) {
return `${baseUrl}${API_ENDPOINT_WEBHOOKS}/container-automation/${token}`;
}
export function baseStackWebhookUrl() {
return `${baseUrl}${API_ENDPOINT_STACKS}/webhooks`;
}
@@ -1,10 +1,12 @@
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { HttpResponse, http } from 'msw';
import { withTestRouter } from '@/react/test-utils/withRouter';
import { withTestQueryProvider } from '@/react/test-utils/withTestQuery';
import { withUserProvider } from '@/react/test-utils/withUserProvider';
import { server } from '@/setup-tests/server';
import { containerAutomationWebhookUrl } from '@/portainer/helpers/webhookHelper';
import { AutoUpdatePanel } from './AutoUpdatePanel';
@@ -113,4 +115,126 @@ describe('AutoUpdatePanel', () => {
// Falls back to the default rollback timeout when missing.
expect(screen.getByLabelText(/Rollback timeout/i)).toHaveValue('120s');
});
function seedSettings(token?: string, enabled = true) {
server.use(
http.get('/api/settings', () =>
HttpResponse.json({
ContainerAutomation: {
AutoHeal: { Enabled: false, CheckInterval: '30s', Scope: 'labeled' },
AutoUpdate: {
Enabled: enabled,
PollInterval: '6h',
Scope: 'labeled',
Cleanup: false,
RollbackOnFailure: false,
RollbackTimeout: '120s',
WebhookToken: token,
},
},
})
)
);
}
it('renders the trigger webhook URL and Regenerate/Clear when a token exists', async () => {
const token = 'abc-123-token';
seedSettings(token);
renderComponent();
const urlField = await screen.findByLabelText(/Update trigger webhook/i);
// The URL is built via the shared helper (sub-path / base-href aware), not a
// bare origin — assert against the same helper.
await waitFor(() =>
expect(urlField).toHaveValue(containerAutomationWebhookUrl(token))
);
expect(
screen.getByRole('button', { name: /Regenerate/i })
).toBeInTheDocument();
expect(screen.getByRole('button', { name: /Clear/i })).toBeInTheDocument();
});
it('warns that the trigger is inactive when a token exists but auto-update is disabled', async () => {
seedSettings('abc-123-token', false);
renderComponent();
// The disabled-state note (a POST would 409) must be shown so the UI does not
// contradict the server.
expect(
await screen.findByText(/trigger is inactive/i)
).toBeInTheDocument();
});
it('does NOT show the inactive note when auto-update is enabled', async () => {
seedSettings('abc-123-token', true);
renderComponent();
// Wait for the section to render, then confirm the note is absent.
await screen.findByLabelText(/Update trigger webhook/i);
expect(screen.queryByText(/trigger is inactive/i)).not.toBeInTheDocument();
});
it('shows Generate and no Clear when no token exists', async () => {
seedSettings(undefined);
renderComponent();
expect(
await screen.findByRole('button', { name: /Generate/i })
).toBeInTheDocument();
expect(
screen.queryByRole('button', { name: /Clear/i })
).not.toBeInTheDocument();
const urlField = screen.getByLabelText(/Update trigger webhook/i);
expect(urlField).toHaveValue('');
});
it('requests a server-side token regeneration without sending a token value', async () => {
seedSettings(undefined);
let putBody: unknown;
server.use(
http.put('/api/settings', async ({ request }) => {
putBody = await request.json();
return HttpResponse.json({});
})
);
renderComponent();
const generate = await screen.findByRole('button', { name: /Generate/i });
await userEvent.click(generate);
await waitFor(() => expect(putBody).toBeDefined());
expect(putBody).toEqual({
ContainerAutomation: { AutoUpdate: { RegenerateWebhookToken: true } },
});
});
it('requests clearing the token', async () => {
seedSettings('some-token');
let putBody: unknown;
server.use(
http.put('/api/settings', async ({ request }) => {
putBody = await request.json();
return HttpResponse.json({});
})
);
renderComponent();
const clear = await screen.findByRole('button', { name: /Clear/i });
await userEvent.click(clear);
await waitFor(() => expect(putBody).toBeDefined());
expect(putBody).toEqual({
ContainerAutomation: { AutoUpdate: { ClearWebhookToken: true } },
});
});
});
@@ -2,9 +2,10 @@ import { RefreshCw } from 'lucide-react';
import { Field, Form, Formik, useFormikContext } from 'formik';
import { notifySuccess } from '@/portainer/services/notifications';
import { containerAutomationWebhookUrl } from '@/portainer/helpers/webhookHelper';
import { Widget } from '@@/Widget';
import { LoadingButton } from '@@/buttons';
import { LoadingButton, Button, CopyButton } from '@@/buttons';
import { FormControl } from '@@/form-components/FormControl';
import { Input, Select } from '@@/form-components/Input';
import { SwitchField } from '@@/form-components/SwitchField';
@@ -68,6 +69,13 @@ export function AutoUpdatePanel() {
>
<InnerForm isLoading={mutation.isLoading} />
</Formik>
<hr />
<WebhookTriggerSection
token={autoUpdate.WebhookToken}
enabled={autoUpdate.Enabled}
/>
</Widget.Body>
</Widget>
);
@@ -95,6 +103,121 @@ export function AutoUpdatePanel() {
}
}
// WebhookTriggerSection renders the inbound registry-push webhook controls: the
// trigger URL (readonly, with a copy button) plus Generate/Regenerate and Clear
// actions. The token itself is server-generated — the client only requests an
// action — so this section never edits the URL, it just displays and rotates it.
function WebhookTriggerSection({
token,
enabled,
}: {
token?: string;
enabled: boolean;
}) {
const mutation = useUpdateSettingsMutation();
// Build the URL via the shared helper so it honours the reverse-proxy sub-path
// (`<base href>`) and the desktop file:// build, rather than a bare origin.
const webhookUrl = token ? containerAutomationWebhookUrl(token) : '';
function regenerate() {
mutation.mutate(
{ ContainerAutomation: { AutoUpdate: { RegenerateWebhookToken: true } } },
{
onSuccess() {
notifySuccess('Success', 'Update trigger webhook token generated');
},
}
);
}
function clear() {
mutation.mutate(
{ ContainerAutomation: { AutoUpdate: { ClearWebhookToken: true } } },
{
onSuccess() {
notifySuccess('Success', 'Update trigger webhook disabled');
},
}
);
}
return (
<div className="form-horizontal">
<div className="form-group">
<div className="col-sm-12">
<TextTip color="blue">
Configure this URL as a push/package webhook on your registry (e.g.
Gitea &quot;Package&quot; webhook or a GHCR{' '}
<code>registry_package</code> webhook, or a{' '}
<code>curl -X POST</code> step at the end of your publish workflow).
Any POST to it immediately runs a full auto-update pass instead of
waiting for the next poll. The token is secret treat the URL as a
credential.
</TextTip>
</div>
</div>
{token && !enabled && (
<div className="form-group">
<div className="col-sm-12">
<TextTip color="orange">
The trigger is inactive while container auto-update is disabled
a POST to this URL returns HTTP 409 and runs no pass. Enable
auto-update above to activate it. The token is kept and can still
be rotated or cleared here.
</TextTip>
</div>
</div>
)}
<FormControl label="Update trigger webhook" inputId="autoupdate_webhook_url">
<div className="flex items-center gap-2">
<Input
id="autoupdate_webhook_url"
readOnly
value={webhookUrl}
placeholder="No webhook token generated"
data-cy="settings-autoUpdateWebhookUrl"
/>
<CopyButton
copyText={webhookUrl}
data-cy="settings-autoUpdateWebhookCopy"
>
Copy
</CopyButton>
</div>
</FormControl>
<div className="form-group">
<div className="col-sm-12 flex gap-2">
<LoadingButton
type="button"
color="default"
isLoading={mutation.isLoading}
loadingText="Saving..."
onClick={regenerate}
data-cy="settings-autoUpdateWebhookRegenerate"
>
{token ? 'Regenerate' : 'Generate'}
</LoadingButton>
{token && (
<Button
type="button"
color="dangerlight"
disabled={mutation.isLoading}
onClick={clear}
data-cy="settings-autoUpdateWebhookClear"
>
Clear
</Button>
)}
</div>
</div>
</div>
);
}
function InnerForm({ isLoading }: { isLoading: boolean }) {
const { values, setFieldValue, isValid, errors } = useFormikContext<Values>();
@@ -36,8 +36,18 @@ export async function getSettings() {
type OptionalSettings = Omit<Partial<Settings>, 'Edge' | 'ContainerAutomation'> & {
Edge?: Partial<Settings['Edge']>;
// ContainerAutomation blocks (AutoHeal / AutoUpdate) are saved independently
// by their own settings panels, so each may be sent on its own.
ContainerAutomation?: Partial<Settings['ContainerAutomation']>;
// by their own settings panels, so each may be sent on its own. AutoUpdate also
// accepts the write-only webhook-token actions (regenerate / clear); the token
// value itself is never sent from the client, only the requested action.
ContainerAutomation?: Omit<
Partial<Settings['ContainerAutomation']>,
'AutoUpdate'
> & {
AutoUpdate?: Partial<Settings['ContainerAutomation']['AutoUpdate']> & {
RegenerateWebhookToken?: boolean;
ClearWebhookToken?: boolean;
};
};
};
export async function updateSettings(settings: OptionalSettings) {
+5
View File
@@ -158,6 +158,11 @@ export interface AutoUpdateSettings {
// previous image when the new container does not become healthy in time.
RollbackOnFailure: boolean;
RollbackTimeout: string;
// WebhookToken is the secret path segment of the inbound registry-push webhook
// (POST /api/webhooks/container-automation/{token}) that triggers an immediate
// update pass. Server-generated only; empty/undefined means the endpoint is
// disabled. Returned only in the admin GET /settings, never in public settings.
WebhookToken?: string;
}
// NotificationSettings holds the per-mechanism webhooks for container-automation