convoy docs
Search

Sync waves

How convoy orders the resources of one Release, and the Releases on one cluster.

Sync waves set the order in which convoy applies resources. convoy orders on two axes. Axis A orders the resources of one Release. Axis B orders the Releases that target one cluster. Both axes use the same three strategies: waitForApplied, waitForHealthy and waitForHealthyWithTimeout.

Axis A: resources in a Release#

Annotations#

convoy reads three annotations on each rendered resource:

AnnotationValueIf not set or not valid
convoy.kumodo.io/sync-waveA signed integerWave 0
convoy.kumodo.io/sync-strategywaitForApplied, waitForHealthy or waitForHealthyWithTimeoutwaitForApplied
convoy.kumodo.io/sync-timeoutA Go duration, for example 2m30s

A value that is not valid does not cause an error. convoy uses the default value.

You can write the annotations in the source. You can also set them with a syncWave mutation. This mutation writes the annotations on each resource that its target matches:

yaml
mutations:
  - type: syncWave
    waveStrategy:
      wave: 1
      strategy: waitForHealthy
    target:
      version: v1
      kind: Service

If a syncWave mutation has no timeout, convoy does not write sync-timeout. A wave with waitForHealthyWithTimeout then gets the default time limit of 30s.

Apply order#

convoy groups the resources by wave number. It applies the waves in ascending order. It applies all resources of a wave before it starts the next wave.

At each reconcile, convoy applies all waves again, from the lowest wave. Thus, convoy also corrects drift while a rollout progresses.

Health gates#

The strategy annotation is on the resources of the wave that waits. A wave holds itself until the wave immediately before it is healthy:

  • waitForApplied (default): the wave does not wait. convoy applies it immediately after the previous wave.
  • waitForHealthy: the wave waits until all resources of the previous wave are Healthy. There is no time limit.
  • waitForHealthyWithTimeout: the wave waits as with waitForHealthy, but with a time limit.

A wave has a health gate when at least one of its resources has waitForHealthy or waitForHealthyWithTimeout. The gate is bounded when at least one resource has waitForHealthyWithTimeout. The time limit is the largest sync-timeout of these resources.

A wave depends only on the wave immediately before it. It does not depend on lower waves. For example, wave 2 has waitForHealthy. Wave 1 is healthy and wave 0 is Degraded. convoy applies wave 2.

The first wave never waits. A Release without spec.healthAssessment has no health bar, so no wave waits. See Health assessment.

Resource waves Pick a scenario and step through time. Each step is an observer pass, then a reconcile. The scenarios assume a health deadline that is longer than the scenario.
Wave 0 stateTime since wave 0 startedAtWave 1Applied reason
HealthyanyappliedApplied
Progressing2m or lessheldWaitingForWaveHealth
Progressingmore than 2mfailedWaveFailed, RolloutFailed or RolledBack
Unknown or not assessed2m or lessheldWaitingForWaveHealth

Wave 1 has a waitForHealthyWithTimeout gate with a 2m limit.

Hold#

While a wave holds, convoy does these things:

  • It sets Applied to False with reason WaitingForWaveHealth.
  • It does not apply the waiting wave or the waves after it.
  • It does not prune.
  • It does not change status.lastSuccessful.
  • It reconciles again later. The observer also sends a /sync request when the highest applied wave becomes healthy.

Timeout#

The time limit starts at status.waves[].startedAt of the previous wave. This is the time of the first apply of that wave. When the time limit passes and the previous wave is not healthy, the wave fails. The result depends on spec.failurePolicy:

spec.failurePolicyResult
Not setApplied=False, reason WaveFailed.
rollback: OnWaveFailure, and status.lastHealthy is setconvoy applies the last healthy render. The rollout is RolledBack.
Other valuesApplied=False, reason RolloutFailed. The rollout is Failed.

If the rollback with OnWaveFailure cannot occur, for example without a render store, the result is RolloutFailed. See Progressive rollout.

A WaveFailed or RolloutFailed result starts the syncFailed lifecycle hooks.

status.lastAttempted.failure has the stage and the message of the failure. It does not have the wave number.

status.waves#

status.waves has one entry for each wave that convoy applied in the last reconcile:

yaml
status:
  waves:
    - number: 0
      resources: 3
      applied: true
      healthy: true
      state: Completed
      startedAt: "2026-11-01T10:00:00Z"
      completedAt: "2026-11-01T10:00:00Z"
  • healthy is true when all resources of the wave are Healthy. It is false when a resource has another state. It is not set while a resource has no state yet.
  • state is always Completed. The other values of the enum are not used.
  • startedAt and completedAt keep the time of the first apply of that wave number.
  • A wave that holds, and the waves after it, have no entry.

Axis B: Releases on a cluster#

waveStrategy#

spec.syncPolicy.waveStrategy sets the position of a Release among the Releases that target the same Cluster:

yaml
spec:
  syncPolicy:
    waveStrategy:
      wave: 1
      strategy: waitForHealthy
  • wave is the cluster wave of the Release. The default is 0.
  • strategy is the bar that the lower-wave Releases must pass. The default is waitForApplied.
  • timeout sets a time limit. It applies only with waitForHealthyWithTimeout.

Predecessors#

The predecessors of a Release are all other Releases that target the same Cluster with a lower wave. This is not only the wave immediately before. Releases with the same wave do not wait for each other. convoy considers only Releases that exist.

A Release must reference a Cluster in its own namespace, because cross-namespace references are rejected. Thus, in practice, the predecessors are in the same namespace.

Hold#

Before each apply, convoy checks the predecessors:

  • With waitForApplied, each predecessor must have Applied=True.
  • With waitForHealthy or waitForHealthyWithTimeout, each predecessor must have Healthy=True.

If a predecessor does not pass, convoy does not apply the Release. It sets Applied to False with reason WaitingForClusterWave. The message names the predecessors that do not pass.

convoy does this check at each reconcile, also after the first apply. A Release stops its drift correction when a predecessor loses its Applied or Healthy state.

Timeout#

Without timeout, a Release waits without a time limit. With waitForHealthyWithTimeout and a timeout, the time limit starts at the last change of the Applied condition of the predecessors. When the time limit passes, convoy sets Applied to False with reason ClusterWaveFailed.

ClusterWaveFailed does not start syncFailed hooks, and it does not start a rollback.

Fan-out#

When the Applied or Healthy status of a Release changes, convoy reconciles each Release with a higher wave on the same cluster immediately. This only starts the check. The check decides if the Release continues.

Release waves on one cluster Three Releases on one Cluster. Pick a scenario and step through time. Each step is a health pass, then a reconcile of each Release.
ReleasewavestrategyWaits forBar
crds0anynothingnone
operator1waitForAppliedcrdsApplied=True
app2waitForHealthycrds and operatorHealthy=True

Three Releases on one Cluster.

Order with lifecycle hooks#

Axis B runs first. Then convoy runs the preSync hooks, then the Axis A waves, then the postSync hooks. See Lifecycle hooks.

In the code

convoy v0.9.0