Create Kubernetes Controller using Rego and MetaController

The declarative approach

Kubernetes Controller

The Kubernetes Controllers are processes which manage resources for Kubernetes.
A Kubernetes Controller watches the changes of resources and generates actions as responses.

If we look closely at the Controller, we will find it takes the Metadata, Spec, and Status of the resource as input. Also, we can see that it generates actions like external actions to other systems and/or CRUD actions to (other) Kubernetes resources.
We can skip the external systems in this discussion. Without external actions, a controller in Kubernetes can be treated as a “function”. The function takes the current cluster state as input and generates the target state as output. Such functions are state transit functions.

MetaController

The MetaController implements the idea of abstracting the controller as a state transit function. It works as a component in the controller. It does all the dirty jobs like communicating with the Kubernetes API, watching the cluster state and executing the actions. The state transition jobs are done by another program through a webhook.

The webhook

As discussed, the webhook should implement the state transition function of a Kubernetes controller. There are lots of choices to develop such webhooks. For example, a python flask app implementing a RESTful interface can be used. Another choice is jsonnet, which is a json template language. The Rego language from the Open Policy Agent project is also a good choice in developing webhooks for MetaController.
The Rego language can take a json or yaml document as input and output a transformed json. The transformation is declarative so that the controller developers can focus on what state should return rather than how the transform should be executed.
Also, the Rego language has a built-in RESTful interface, so that it can be integrated easily.
Hence the architecture would look like this:

An example

This k8s-transient-role-binding is an attempt to implement a Kubernetes controller using MetaController and Rego.
The controllers provide a limited lifetime RoleBinding and ClusterRoleBinding mechanism for Kubernetes. To achieve this, two additional fields are added in comparison to the RoleBinding and ClusterRoleBinding objects: they are validUntil and validFrom. The controller creates the corresponding RoleBinding(or ClusterRoleBiding) objects when the current time fits in validFrom and validUntil and deletes them otherwise.
The MetaController declaration of this controller is defined in the "controller/transient-rolebinding-metacontroller.yaml" file; the most important parts are as follows.
First, the resources part:

spec:
  parentResource:
    apiVersion:  example.com/v1
    resource: transientrolebindings
    revisionHistory:
      fieldPaths: []
  childResources:
  - apiVersion: rbac.authorization.k8s.io/v1
    resource: rolebindings
    updateStrategy:
      method: Recreate
  generateSelector: true
  resyncPeriodSeconds: 300

This snippet means that the controller manages parent resource: transientrolebinding and child resource: rolebinding. The controller will poll the status of both parent and child resources every 300 seconds.
And the webhook.

hooks:
    sync:
      webhook:
        url: http://transientresourcecontroller-webhook.metacontroller-webhooks:8181/v0/data/controller
        timeout: 10s

The webhook is a Kubernetes service pointing to the rego deployment; the Rego-related objects are placed in “controller/opa-webhook/”.

The rules are not complex:

# Copyright 2022 Google LLC

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

#     https://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
package controller

rfc3339time(ns) = concat("T", [
        sprintf("%04d-%02d-%02d", time.date(ns)),
        sprintf("%02d:%02d:%02dZ", time.clock(ns)),
])

status := {"conditions": array.concat(
        [x |
                not count(children) == count(input.children_[])
                count(children) < 2
                x := {
                        "lastTransitionTime": rfc3339time(time.now_ns()),
                        "status": ["False", "True"][count(children)],
                        "type": "effective",
                }
        ],
        [input.parent.status.conditions[x] | input.parent.status.conditions[x]],
)}

children[child] {
        time.now_ns() > time.parse_rfc3339_ns(input.parent.validFrom)
        time.now_ns() < time.parse_rfc3339_ns(input.parent.validUntil)
        child := {
                "apiVersion": input.controller.spec.childResources[0].apiVersion,
                "kind": split({x | input.children[x]}_[], ".")[0],
                "metadata": {"name": input.parent.metadata.name},
                "roleRef": input.parent.roleRef,
                "subjects": input.parent.subjects,
        }
}

The output contains only two fields: status and children, each of them are managed by a rule starting with the respective words. Specifically, according to the rule, the children field will contain a RoleBinding object when the current time fits in the validFrom and validUntil or an empty list otherwise.

In the next posts, I will discover more possibilities with this MetaController and Rego combination.