Sync waves
How convoy orders the resources of one Release, and the Releases on one cluster.
Sync waves set the order in which convoy applies resources. convoy orders on two axes. Axis A
orders the resources of one Release. Axis B orders the Releases that target one cluster. Both
axes use the same three strategies: waitForApplied, waitForHealthy and
waitForHealthyWithTimeout.
Axis A: resources in a Release#
Annotations#
convoy reads three annotations on each rendered resource:
| Annotation | Value | If not set or not valid |
|---|---|---|
convoy.kumodo.io/sync-wave | A signed integer | Wave 0 |
convoy.kumodo.io/sync-strategy | waitForApplied, waitForHealthy or waitForHealthyWithTimeout | waitForApplied |
convoy.kumodo.io/sync-timeout | A Go duration, for example 2m | 30s |
A value that is not valid does not cause an error. convoy uses the default value.
You can write the annotations in the source. You can also set them with a syncWave
mutation. This mutation writes the annotations on each resource that its
target matches:
mutations:
- type: syncWave
waveStrategy:
wave: 1
strategy: waitForHealthy
target:
version: v1
kind: ServiceIf a syncWave mutation has no timeout, convoy does not write sync-timeout. A wave with
waitForHealthyWithTimeout then gets the default time limit of 30s.
Apply order#
convoy groups the resources by wave number. It applies the waves in ascending order. It applies all resources of a wave before it starts the next wave.
At each reconcile, convoy applies all waves again, from the lowest wave. Thus, convoy also corrects drift while a rollout progresses.
Health gates#
The strategy annotation is on the resources of the wave that waits. A wave holds itself until the wave immediately before it is healthy:
waitForApplied(default): the wave does not wait. convoy applies it immediately after the previous wave.waitForHealthy: the wave waits until all resources of the previous wave areHealthy. There is no time limit.waitForHealthyWithTimeout: the wave waits as withwaitForHealthy, but with a time limit.
A wave has a health gate when at least one of its resources has waitForHealthy or
waitForHealthyWithTimeout. The gate is bounded when at least one resource has
waitForHealthyWithTimeout. The time limit is the largest sync-timeout of these resources.
A wave depends only on the wave immediately before it. It does not depend on lower waves. For
example, wave 2 has waitForHealthy. Wave 1 is healthy and wave 0 is Degraded. convoy applies
wave 2.
The first wave never waits. A Release without spec.healthAssessment has no health bar, so no
wave waits. See Health assessment.
Hold#
While a wave holds, convoy does these things:
- It sets
AppliedtoFalsewith reasonWaitingForWaveHealth. - It does not apply the waiting wave or the waves after it.
- It does not prune.
- It does not change
status.lastSuccessful. - It reconciles again later. The observer also sends a
/syncrequest when the highest applied wave becomes healthy.
Timeout#
The time limit starts at status.waves[].startedAt of the previous wave. This is the time of the
first apply of that wave. When the time limit passes and the previous wave is not healthy, the
wave fails. The result depends on spec.failurePolicy:
spec.failurePolicy | Result |
|---|---|
| Not set | Applied=False, reason WaveFailed. |
rollback: OnWaveFailure, and status.lastHealthy is set | convoy applies the last healthy render. The rollout is RolledBack. |
| Other values | Applied=False, reason RolloutFailed. The rollout is Failed. |
If the rollback with OnWaveFailure cannot occur, for example without a render store, the result
is RolloutFailed. See Progressive rollout.
A WaveFailed or RolloutFailed result starts the syncFailed
lifecycle hooks.
status.lastAttempted.failure has the stage and the message of the failure. It does not have the
wave number.
status.waves#
status.waves has one entry for each wave that convoy applied in the last reconcile:
status:
waves:
- number: 0
resources: 3
applied: true
healthy: true
state: Completed
startedAt: "2026-11-01T10:00:00Z"
completedAt: "2026-11-01T10:00:00Z"healthyistruewhen all resources of the wave areHealthy. It isfalsewhen a resource has another state. It is not set while a resource has no state yet.stateis alwaysCompleted. The other values of the enum are not used.startedAtandcompletedAtkeep the time of the first apply of that wave number.- A wave that holds, and the waves after it, have no entry.
Axis B: Releases on a cluster#
waveStrategy#
spec.syncPolicy.waveStrategy sets the position of a Release among the Releases that target the
same Cluster:
spec:
syncPolicy:
waveStrategy:
wave: 1
strategy: waitForHealthywaveis the cluster wave of the Release. The default is0.strategyis the bar that the lower-wave Releases must pass. The default iswaitForApplied.timeoutsets a time limit. It applies only withwaitForHealthyWithTimeout.
Predecessors#
The predecessors of a Release are all other Releases that target the same Cluster with a lower
wave. This is not only the wave immediately before. Releases with the same wave do not wait for
each other. convoy considers only Releases that exist.
A Release must reference a Cluster in its own namespace, because cross-namespace references
are rejected. Thus, in practice, the predecessors are in the same namespace.
Hold#
Before each apply, convoy checks the predecessors:
- With
waitForApplied, each predecessor must haveApplied=True. - With
waitForHealthyorwaitForHealthyWithTimeout, each predecessor must haveHealthy=True.
If a predecessor does not pass, convoy does not apply the Release. It sets Applied to False
with reason WaitingForClusterWave. The message names the predecessors that do not pass.
convoy does this check at each reconcile, also after the first apply. A Release stops its drift
correction when a predecessor loses its Applied or Healthy state.
Timeout#
Without timeout, a Release waits without a time limit. With waitForHealthyWithTimeout and a
timeout, the time limit starts at the last change of the Applied condition of the
predecessors. When the time limit passes, convoy sets Applied to False with reason
ClusterWaveFailed.
ClusterWaveFailed does not start syncFailed hooks, and it does not start a rollback.
Fan-out#
When the Applied or Healthy status of a Release changes, convoy reconciles each Release with
a higher wave on the same cluster immediately. This only starts the check. The check decides if
the Release continues.
Order with lifecycle hooks#
Axis B runs first. Then convoy runs the preSync hooks, then the Axis A waves, then the
postSync hooks. See Lifecycle hooks.
In the code
convoy v0.9.0
internal/waves.Groupinternal/waves/waves.go:103internal/controller/release.(*Controller).stageSyncinternal/controller/release/controller.go:1092internal/controller/release.(*Controller).waveHealthGateinternal/controller/release/controller.go:1593internal/controller/release.precedingWaveHealthyinternal/controller/release/controller.go:2063internal/controller/release.(*Controller).clusterWaveGateinternal/controller/release/controller.go:1968internal/waves.ClusterPredecessorsinternal/waves/cluster.go:68internal/controller/release.ClusterWaveSignalChangedinternal/controller/release/controller.go:2038