Encrypted Mosquitto broker in Kubernetes

Note: an updated version of this article has been posted to Medium

Mosquitto can usually be installed on an Ubuntu server fairly easily using the APT package manager. By Default, the broker handles unencrypted MQTT connections but it can be configured to use SSL certificates obtained, for example, using Certbot and thus enable MQTTs connections. This configuration is usually achieved by editing the Mosquitto configuration file in /etc/mosquitto so as to point to certificates obtained independently. However, when deploying Mosquitto to Kubernetes, one would prefer not to edit configuration files manually after install. Moreover, in Kubernetes, one can use Cert-manager to obtain SSL certificates. Thus, this article presents an efficient method to deploy a secure MQTTs broker in Kubernetes.

Deployment

Let's begin by creating a deployment for the broker. This can be done easily with the following manifest.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mosquitto
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mosquitto
  template:
    metadata:
      labels:
        app: mosquitto
    spec:
      containers:
      - name: mosquitto
        image: eclipse-mosquitto
        ports:
        - containerPort: 8883
        - containerPort: 9001
        volumeMounts:
        - mountPath: /etc/mosquitto/
          name: config
        - mountPath: /mosquitto/certs/
          name: certs
        - mountPath: /mosquitto/data/
          name: data
      volumes:
      - name: config
        configMap:
          name: mosquitto-config
      - name: certs
        secret:
          secretName: mosquitto-certs
      - name: data
        persistentVolumeClaim:
          claimName: mosquitto-data

The volumeMounts are the essential part to pay attention to. Here, we have 3 volumes mounted:

• /mosquitto/data: which holds the Mosquitto persistent data, mapped to a PVC

• /mosquitto/certs: which will hold SSL certificates, mapped to a secret

• /etc/mosquitto/: which is normally where the Mosquitto configuration file resides, mapped to a configmap

PVC for Persistent data

The Mosquitto persistent data should survive even when pods are deleted or recreated. Consequently, the persistent data is stored in a PV, for which this would be a typical PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mosquitto-data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

Secret for SSL certificates

The Mosquitto deployment specifies that /mosquitto/certs is to be mounted from a secret. This secret is in fact that created automatically by Cert-manager when creating an Ingress for our Broker:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mosquitto
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - mqtt.example.com
    secretName: mosquitto-certs
  rules:
  - host: mqtt.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: mosquitto-clusterip
            port: 
              number: 8883

Configmap for Mosquitto configuration

The last volume to take care of is that of the Mosquitto configuration. The Mosquitto configuration is usually achieved by editing a text file but here, the file actually mounted as a configmap. The configuration specifies the path to the SSL certificates which have been mounted as a secret:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mosquitto-config
data:
  mosquitto.conf: |

    persistence true
    persistence_location /mosquitto/data/

    log_dest stdout

    # MQTT with TLS (MQTTS)
    listener 8883
    protocol mqtt

    cafile /etc/ssl/certs/ca-certificates.crt
    keyfile /mosquitto/certs/tls.key
    certfile /mosquitto/certs/tls.crt

Services

Finally, we add a service to connect the ingress to the deployment:

apiVersion: v1
kind: Service
metadata:
  name: mosquitto-mqtts
spec:
  type: ClusterIP
  selector:
    app: mosquitto
  ports:
  - port: 8883