Components

convoy has three server binaries and one kubectl plugin. Each binary is a separate process. The binaries do not call each other directly, with one exception: the apiserver sends some subresource requests to the observer. All other communication goes through the convoy API objects on the host cluster.

Summary#

The flags of each binary are on Binaries and flags.

convoy-apiserver#

convoy-apiserver is an aggregated API server. It uses the k8s.io/apiserver library. It serves three API groups at version v1alpha1:

• core.convoy.kumodo.io: Cluster, Release, ReleaseInventory.
• source.convoy.kumodo.io: GitRepository, HelmRegistry, OCIRegistry.
• extensions.convoy.kumodo.io: GlobalHealthRuleSet, HealthRuleSet.

The objects are not CRDs. The host kube-apiserver forwards requests for these groups to convoy-apiserver through one APIService object for each group. The development manifests register v1alpha1.core.convoy.kumodo.io, v1alpha1.source.convoy.kumodo.io and v1alpha1.extensions.convoy.kumodo.io. They point to the convoy-apiserver Service in convoy-system.

The apiserver uses the recommended options of the library:

• Storage. The apiserver keeps all objects in etcd under the prefix /registry/convoy.kumodo.io. You give the etcd connection with the standard --etcd-* flags. convoy does not embed etcd.
• Authentication and authorization. The apiserver delegates both to the host cluster. The host cluster RBAC controls each resource and each subresource, for example releases/sync.
• Admission. The apiserver runs the admission chain of the library. It disables the MutatingAdmissionPolicy plugin.
• Shard label. When a user writes a Release, Cluster or source object, the apiserver sets the shard label. The --shards flag must have the same value as in the controllers. See Shard-scoped caches.

The apiserver handles these subresources itself: status, sync, pin, mutations and lifecycle. A POST to releases/<name>/sync reads the Release, sets the annotation convoy.kumodo.io/sync-requested-at and updates the Release in etcd. With force=true, it also sets convoy.kumodo.io/sync-force-requested-at.

The apiserver does not render and does not connect to target clusters. It sends GET requests for preview, diff, health and history to the observer at <--observer-url>/<subresource>/<namespace>/<name>. It copies the query string. The default URL is http://convoy-observer.convoy-system.svc:8080. An empty --observer-url makes these subresources return 503 Service Unavailable. See Release subresources.

convoy-controller-manager#

The controller-manager connects to the host kube-apiserver with --kubeconfig or with the in-cluster configuration. It uses two clients:

• The convoy clientset, for the convoy API objects.
• The Kubernetes clientset, for Secrets, ConfigMaps and the sharding Leases.

It runs five controllers on a shared work-queue harness (internal/controller/base):

The source controllers only check that a source is reachable. They do not fetch content. The Release controller fetches content on demand through a fetch coordinator. The coordinator keeps the content in an in-memory artifact store. The process loses this store at a restart and fetches the content again. See Sources and Rendering.

With --render-store-url, the controller-manager writes each render to an object store before it applies the render. See Render storage.

The controller-manager adds three event handlers to its informers:

• A Release update that changes the status of Applied or Healthy enqueues the Releases of a higher cluster wave on the same cluster.
• A ReleaseInventory update that changes the health of an entry enqueues the Release with the same name.
• A change of shard ownership enqueues all owned objects of all five controllers.

convoy-observer#

The observer connects to the host kube-apiserver in the same way as the controller-manager. It runs two controllers on the same harness and one HTTP server:

• Drift loop (observer.Controller). It renders the desired state, compares it with the live objects and writes the InSync condition. In automatic sync mode, it sends POST /sync when it finds drift. See Drift.
• Health loop (observer.HealthController). It assesses the objects in the ReleaseInventory, writes the per-object health to the ReleaseInventory and writes the Healthy condition. It also polls lifecycle hook workloads. See Health loop.
• HTTP server on --serve-addr (default :8080). It serves GET /preview/, /diff/, /health/ and /history/ followed by {namespace}/{name}. Only the apiserver must reach it. See Security.

The observer renders independently of the controller-manager. It has its own fetch coordinator and its own in-memory artifact store. With --render-store-url, it reads stored renders for history, the historical diff, a pinned render and a rolled-back render. It never writes to the render store.

The observer never applies an object to a target cluster. For the drift check, it sends a server-side apply dry run with the field manager convoy.

Target clusters#

The controller-manager and the observer both connect to target clusters through the clusterconn.Connector. For each request, the connector builds a REST configuration from the Cluster object and returns a dynamic client and a REST mapper. The inCluster mode uses the configuration of the process. The agentless mode uses the authenticator for spec.agentless.auth.type. In v0.9.0, the only registered type is kubeconfig. See Clusters and Cluster auth providers.

The agent connection mode is planned. Not built

kubectl convoy#

kubectl convoy is a kubectl plugin. It loads the kubeconfig with the standard client loading rules. It sends requests to the host kube-apiserver, which forwards them to convoy-apiserver. It has no other connection. See CLI.

Watches and caches#

The controller-manager and the observer read objects from informer caches, not from etcd directly. The default resync period is 2 minutes (--resync).

• The Release, ReleaseInventory and Cluster caches can be shard-scoped (--shard-scoped-cache).
• The source caches and the rule-set caches always contain all objects, because a Release can refer to an object of any shard.

The two binaries shard their work independently. The controller-manager uses the Lease component convoy-cm, and the observer uses convoy-obs. See Sharding.