Day 11 - Going from local Docker-compose to Kubernetes

Written by: Sebastien Goasguen (@sebgoa)
Edited by: Daniel Kang (@kangman)

In this Sysadvent blog we will look at a neat tool called kompose which is part of the Kubernetes incubator. Kompose lets you go easily from Docker-compose application manifests to Kubernetes.

Docker has a great user experience, developers can quickly install the Docker engine on their local machine and get started pulling images, building their own and running containers. After building a few images and to avoid writing bash wrappers to launch multiple containers, Compose enters your arsenal. With it you can write a YAML file that describes the several containers you want to manage to run your distributed application.

A quick look at Compose

Docker compose is again easy to install on your local machine. The format of the Compose files is also very intuitive. For example to launch a single container, the simplest Compose file you could write would be something like:

version: '2'

services:
   redis:
     image: redis

You see a version being defined (yes, there was a version 1), and then a list of services. In this example we only have one service called redis, that service is defined by a single Docker image: redis which will be pulled form the Docker Hub.

To launch this toy Compose application, you use the docker-compose CLI and run docker-compose up. This is quite handy, all the parameters that you use for your docker run commands can be specified in the YAML manifest, and it is better than writing your own BASH wrappers.

Enters Kubernetes

However, what happens if you want to take this distributed application and start it on a Kubernetes cluster ? Kubernetes is quickly becoming the container orchestration of choice, and a great alternative to Docker Swarm. In this post we will not go into pros and cons of both system and just consider that you may be interested in using Kubernetes in your data-center, while developers will have created their application using Docker compose.

To move to Kubernetes, you could start from scratch, learn the API and the specifications, or you could use kompose and save yourself some time.

In Kubernetes, to keep it simple, applications are made of a set of deployments and services. A deployment is a declarative manifest that tells the cluster what should be running. A service is a network abstraction to provide network ingress to your containers. Both of these are defined either in JSON or YAML and created via the API server using the kubectl Kubernetes client.

If you want to get started quickly with Kubernetes, you should try out minikube. With an endpoint handy and the CLI installed, starting redis like we did in the previous section would be a single command:

kubectl run redis --image=redis

However your real application is probably more complex than a single container image, and your Compose file will be longer and more detailed.

That’s where kompose becomes handy, it automatically transforms your Compose file into Kubernetes deployments and services and calls the API server to launch the application. Let’s have a closer look.

Convert Compose format to Kubernetes manifests

In recent Docker events, the Docker Voting App has become quite popular to show case Docker’s ability to build, ship and run. It is made of five containers, each written in a different language. A frontend allows you to cast a vote, another frontend allows you to see the result. In the backend, a redis queue collects the votes, a worker stores them in a database. This is a show-case application, by no means does this represent best practice for such an application !

To launch it, a Docker compose file could look like this:

version: "2"

services:
  vote:
    image: docker/example-voting-app-vote:latest
    labels:
     - "com.example.description=Vote"
     - "kompose.service.type=nodeport"
    ports:
      - "5000:80"

  redis:
    image: redis:alpine
    ports: ["6379"]

  worker:
    image: docker/example-voting-app-worker:latest

  db:
    image: postgres:9.4
    ports: ["5432"]
    labels:
     - "com.example.description=Postgres Database"

  result:
    image: tmadams333/example-voting-app-result:latest
    ports:
      - "5001:80"
    labels:
    - "kompose.service.type=nodeport"

Note that in this example we used pre-built images. Compose can build images on the fly if you have the source code and the required Dockerfiles. Also note that we set some specific labels for kompose to be able to give extrenal access to the web frontends of the application.

But how can you easily run this on Kubernetes ? The answer is to use kompose.

A single command similar to docker-compose will start the application on Kubernetes:

kompose -f docker-voting.yml up

This will transform each service into a Kubernetes deployment object, expose it via a Kubernetes service object and launch everything on your cluster. You will then be able to manage the application with the regular Kubernetes CLI kubectl, for example listing the deployments:

$ kubectl get deployments
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
db        1         1         1            1           38m
redis     1         1         1            1           38m
result    1         1         1            1           38m
vote      1         1         1            1           38m
worker    1         1         1            1           38m

You will then be able to access the vote and result frontend through their Kubernetes services, and decide which your prefer, Cats or Dogs.

To do this, use the kubectl describe command and list the the vote and result services. You will see a random high port in the NodePort section. Open your browser on one of the nodes of your Kubernetes cluster (i.e kubectl get nodes) specifying those ports and you will see the web frontends. For example, the sample below shows that the vote application is available on port 31940:

$ kubectl describe svc vote
Name:   vote
Namespace:  default
Labels:   service=vote
Selector:  service=vote
Type:   NodePort
IP:   10.0.155.137
Port:   5000 5000/TCP
NodePort:  5000 31940/TCP
Endpoints:  <none>
Session Affinity: None

Changing kompose behavior

This up command can appear quite magical, if you want to change this behavior, check the usage of kompose. You will see that you can use the convert sub commands with various flags. I won’t cover all of them, but a few things are quite neat.

First you can output the converted objects to stdtout and specify whether you like JSON or YAML.

kompose -f docker-voting.yml --stdout --yaml
apiVersion: v1
items:
- apiVersion: v1
  kind: Service
  metadata:
    annotations:
      com.example.description: Postgres Database
    creationTimestamp: null
...

And if you know Kubernetes objects you can change the default behavior and instead of creating deployments, you can create DaemonSets, ReplicationControllers, or Charts.

kompose -f docker-voting.yml convert --daemonset --stdout --yaml
...
- apiVersion: extensions/v1beta1
  kind: DaemonSet
  metadata:
    annotations:
      com.example.description: Postgres Database
    creationTimestamp: null
    name: db
  spec:
    template:
...

Wrapping up

On your journey to using containers, you will most likely develop Docker compose files. There is currently no standard to define containerized distributed applications, therefore tools like kompose which allow easy transformation of application manifests between different container orchestrators are going to be quite useful.

With kompose you can quickly start experimenting with Kubernetes and learn the different API objects that make up this powerful API. Give it a Shot !