Deletion
What convoy does when you delete a Release or a Cluster.
When you delete a Release, convoy deletes the objects of the Release from the target cluster
before the API server deletes the Release. A finalizer holds the Release until this cleanup is done.
spec.deletionPolicy and the delete-phase lifecycle hooks control the cleanup.
convoy adds the finalizer release.core.convoy.kumodo.io/prune-workloads at the first reconcile
of each Release.
Check the target cluster#
convoy resolves the Cluster only when it must delete objects or run delete hooks. It reads
status.reachable of the Cluster. It does not connect to the cluster for this check.
State of the Cluster | blockOnClusterUnreachable: true | blockOnClusterUnreachable: false |
|---|---|---|
| Reachable | Steps 2 to 4 run. | Steps 2 to 4 run. |
| Not reachable | The deletion waits. convoy retries. | convoy skips steps 2 to 4. |
| Does not exist | convoy skips steps 2 to 4. | convoy skips steps 2 to 4. |
When convoy skips steps 2 to 4, the objects stay on the cluster. convoy still deletes the
ReleaseInventory, so it does not know about these objects after the deletion.
Delete hooks#
convoy runs the preDelete and postDelete hooks as Jobs or Pods on the target cluster. While a
hook runs, convoy checks it again every 10 seconds. The observer also wakes the controller when a
batch of hooks completes. See Lifecycle hooks.
- preDelete. A failed hook with
onFailure: Failblocks the deletion. The finalizer stays, and convoy does not delete objects. A failed hook withonFailure: Continuedoes not block. - postDelete. These hooks run after convoy deleted the objects. A failed hook does not block the deletion, for each
onFailurevalue.
apiVersion: core.convoy.kumodo.io/v1alpha1
kind: Release
metadata:
name: podinfo-in-cluster-helm-with-deletion-lifecycle
namespace: default
lifecycle:
preDelete:
- name: pre-sleep
type: Job
timeout: 30s
onFailure: Continue
cleanup: OnSuccess
jobTemplate:
metadata:
labels:
test-label: "true"
annotations:
test-annotation: "true"
spec:
template:
spec:
restartPolicy: Never
containers:
- name: job-worker
image: "busybox:1.38.0"
command: ["sh", "-c", "sleep 10"]
postDelete:
- name: post-sleep
type: Job
timeout: 30s
onFailure: Continue
cleanup: OnSuccess
jobTemplate:
metadata:
labels:
test-label: "true"
annotations:
test-annotation: "true"
spec:
template:
spec:
restartPolicy: Never
containers:
- name: job-worker
image: "busybox:1.38.0"
command: ["sh", "-c", "sleep 10"]
mutations:
- type: syncWave
waveStrategy:
strategy: waitForHealthy
wave: 1
target:
group: ""
version: v1
kind: Service
spec:
clusterRef:
name: dev-local
source:
type: gitRepository
gitRepository:
name: podinfo
namespace: default
ref:
branch: master
content:
type: helm
helm:
path: charts/podinfo
releaseName: podinfo-helm
values:
- type: inline
inline:
probes:
readiness:
initialDelaySeconds: 15
liveness:
initialDelaySeconds: 15
syncPolicy:
automatic: true
prune:
orphaned: false
includeCRDs: false
includeNamespaces: false
interval: 1m
healthAssessment:
globalRuleSetSelector:
matchLabels:
convoy.kumodo.io/default: "true"
deadline: 1m
Delete the objects#
With keepWorkloads: false (the default), convoy deletes each object that the
ReleaseInventory records. An object that is already absent does not cause an error.
The inventory contains only the objects that convoy manages now. An object that a sync with
prune.orphaned: false dropped from the inventory stays on the cluster. See
Apply and prune.
With keepWorkloads: true, the objects stay on the cluster. The delete hooks still run.
Delete the convoy data#
After the objects and the hooks, convoy deletes its own data for the Release:
- The
ReleaseInventory. - All renders and the history of the Release in the render store, if a render store is configured.
- The finalizer. The API server then deletes the Release.
convoy does each step before it clears the finalizer. If a step fails, the finalizer stays and
convoy retries. So no ReleaseInventory or stored render stays without its Release.
A failed step in the deletion flow retries with the default rate limiter of the controller. The
retryBackoff policy does not apply. During the deletion, the observer stops the drift and
health checks of the Release. It continues to watch the delete hooks.
Delete a Cluster#
convoy adds the finalizer cluster.core.convoy.kumodo.io/prune-releases to each Cluster.
spec.deletionPolicy.pruneReleases controls what occurs to the Releases that target it:
false(the default). convoy deletes theClusterat once. The Releases stay, and theirAppliedcondition isFalsewith reasonClusterNotReady.true. convoy deletes each Release in the namespace of theClusterthat references it. TheClusterstays until the API server deletes all those Releases. Each Release runs its own deletion flow, with its owndeletionPolicy.
With pruneReleases: true, the Cluster still exists while its Releases run their deletion
flow. So the Releases can delete their objects from the cluster. See
Cluster.
In the code
convoy v0.9.0
internal/controller/release.(*Controller).finalizeinternal/controller/release/controller.go:553internal/controller/release.(*Controller).clusterForFinalizeinternal/controller/release/controller.go:646internal/controller/release.(*Controller).pruneRefsinternal/controller/release/controller.go:666internal/controller/release.(*Controller).runHookPhaseinternal/controller/release/hookphase.go:62internal/controller/cluster.(*Controller).finalizeinternal/controller/cluster/controller.go:132internal/controller/cluster.(*Controller).pruneReleasesinternal/controller/cluster/controller.go:164