Kubernetes Operator with Java and Quarkus

Originally published by Lunatech by Ihor Mutel, read the full article here

A Kubernetes Operator, referred to as an operator in the remainder of this article, can be used for various purposes. The most common ones are managing database clusters, preparing database schemas, and restoring backups. Even though Kubernetes provides extensive support for multiple container management operations, it doesn’t cover all potential cases related to application management. Some applications need the manual intervention of a human operator. A Kubernetes operator is an application that runs in the cluster and manages resources. It allows the transfer of operational knowledge of the human operator into software components. Operators can be used for multiple purposes such as restoring data, creating cloud resources, resilience testing, and many others.

The purpose of this blog post is to demonstrate how to quickly write and deploy a simple Kubernetes operator using Java, and provide an example of tasks that can be performed using an operator in the context of the deployment of a service built with Quarkus.

I decided to use a Quarkus service written in Java for the following reasons:

  • Being able to develop operators in Java can bring a lot of benefits for companies and individuals that are focused mainly on the Java stack.
  • Technologies such as containers, microservices, immutable infrastructure, deployed via declarative code are common elements of cloud-native computing approach. An Operator needs to be deployed in a container, and the state of resources managed by an operator is defined in a declarative way. Quarkus with the main focus on cloud-native, and microservice solutions looks like a good fit for building an operator.
  • The operator SDK also has neat features such as generating Kubernetes resources for operator deployments and creating custom resource definition yaml files based on the Java code.
  • The Java kubernetes-client supports multiple operations: such as creating resources, patching, copying files, executing commands, fetching logs and many others.

WHAT THIS OPERATOR DOES?

This simple hello world operator demonstrates how to create a stateful set via Kubernetes API and make sure that the data is loaded into each pod and consumed by the application before creating new pods.

image 2021 04 18 22 56 52 151

It performs the following operations with Kubernetes API using the java kubernetes-client and quarkus-operator-sdk :

  • watches custom resource creation/modification
  • watches pod creation/modification
  • creates and scales statefulsets
  • checks content of log output
  • executes commands in a container

PROJECT SETUP

Quarkus project provides developers with tools that allow bootstrapping an application without spending a lot of time on boilerplate code preparation https://code.quarkus.io. The generated project also includes a Dockerfile for building Docker images.

Dockerfile

ADD DEPENDENCIES INTO A PROJECT:

The next step is to add following dependencies:

Quarkus Extension to write operators in Java.

quarkus-operator-sdk

<dependency>
<groupId>io.quarkiverse.operatorsdk</groupId>
<artifactId>quarkus-operator-sdk</artifactId>
<version>1.8.2</version>
</dependency>

Kubernetes Java Client for interacting and managing kubernetes resources via REST API. In recent years, this client has evolved to a full-fledged alternative to the kubectl command-line tool for Java-based development.

kubernetes-client

<dependency>
 <groupId>io.kubernetes</groupId>
 <artifactId>client-java</artifactId>
 <version>12.0.0</version>
</dependency>

To facilitate local development we need to create a proxy connection between Kubernetes cluster API and a local machine.

kubectl --namespace kube-system --disable-filter=true proxy kube-apiserver-kind-control-plane

Additionally, some extra steps are required to configure a local development environment. In the following snippet of code, I disabled certificates validation and provided Kubernetes API endpoint name and default namespace.

quarkus.kubernetes-client.trust-certs=false
quarkus.kubernetes-client.namespace=default
quarkus.kubernetes-client.master-url=http://127.0.0.1:8001/

KUBERNETES CUSTOM RESOURCE DEFINITIONS

It’s possible to extend the Kubernetes API with custom resources in order to store and modify the desired object specification and state.

Based on the content of the classes in the below application, it generates custom-resource-definitions which can be used to create custom-resources. When the application starts it outputs yaml, which is the custom resource definitions to a file in the ./target/kubernetes/ directory.

Create a custom resource definition which extends the existing Kubernetes API.

HelloWorld.java
@Group("example.com")
@Version("v1alpha1")
@ShortNames("hw")
public class HelloWorld extends CustomResource<HelloWorldSpec, HelloWorldStatus> implements Namespaced {

   private HelloWorldSpec spec;
   private HelloWorldStatus status;

   // {...}
}

Creating specification fields of the custom resource.

HelloWorldSpec.java
public class HelloWorldSpec {

   private String name;
   private String image;
   private String data;
   private int replicas;

   // {...}
}

The status fields of the custom resource is used to store the status of the replica set. In this particular case, it is used to count the number of deployed pods.

HelloWorldStatus.java
public class HelloWorldStatus {

   private Integer readyReplicas = 0;

   // {...}
}

Create a custom resource of kind HelloWorld.

hello-world-example.yaml
apiVersion: example.com/v1alpha1
kind: HelloWorld
metadata:
  name: hello-world-example
spec:
  name: hello-world
  image: busybox
  replicas: 3
  data: |
   Example of injected data
kubectl create -f hello-world-example.yaml

A CLIENT TO CONNECT TO THE KUBERNETES API

KubernetesClientProducer.java
@Singleton
public class KubernetesClientProducer {

   @Produces
   public KubernetesClient kubernetesClient() {
       return new DefaultKubernetesClient(command in container);
   }
}

A controller which listens to custom resource creation and updates

@Controller(namespaces = "default")
public class HelloWorldController  implements ResourceController<HelloWorld> {

   / ... /

   @Override
   public DeleteControl deleteResource(HelloWorld resource, Context<HelloWorld> context) {...}

   @Override
   public UpdateControl<HelloWorld> createOrUpdateResource(HelloWorld helloWorldRequest, Context<HelloWorld> context) {...}

   @Override
   public void init(EventSourceManager eventSourceManager) {...}
} Want to continue reading this blog? Look here at the original full version!