This recipe is a work in progress
This recipe is incomplete, and is featured to align the patrons's "premix" repository with the cookbook. "premix" is a private git repository available to all Patreon patrons, which includes all the necessary .yml files for all published recipes. This means that patrons can launch any recipe with just a
git pull and a
kubectl create -f *.yml 👍
So... There may be errors and inaccuracies. Jump into Discord if you're encountering issues 😁
There's currently a mysterious bug though, which prevents TCP communication between Home Assistant and the camera, when MQTT services are enabled on the camera and the mqtt broker runs on the same Raspberry Pi as Home Assistant, using Hass.io.
A workaround to this bug is to run an MQTT broker external to the raspberry pi, which makes the whole problem GoAway™. Since an MQTT broker is a single, self-contained container, I've written this recipe as an introduction to our Kubernetes cluster design.
MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks. The design principles are to minimise network bandwidth and device resource requirements whilst also attempting to ensure reliability and some degree of assurance of delivery. These principles also turn out to make the protocol ideal of the emerging “machine-to-machine” (M2M) or “Internet of Things” world of connected devices, and for mobile applications where bandwidth and battery power are at a premium.
Create data locations¶
Although we could simply bind-mount local volumes to a local Kubuernetes cluster, since we're targetting a cloud-based Kubernetes deployment, we only need a local path to store the YAML files which define the various aspects of our Kubernetes deployment.
We use Kubernetes namespaces for service discovery and isolation between our stacks, so create a namespace for the mqtt stack by creating the following .yaml:
1 2 3 4 5 6 7
cat <<EOF > /var/data/mqtt/namespace.yml apiVersion: v1 kind: Namespace metadata: name: mqtt EOF kubectl create -f /var/data/mqtt/namespace.yaml
Create persistent volume claim¶
Persistent volume claims are a streamlined way to create a persistent volume and assign it to a container in a pod. Create a claim for the certbot data:
1 2 3 4 5 6 7 8 9 10 11 12 13 14
cat <<EOF > /var/data/mqtt/persistent-volumeclaim.yml kind: PersistentVolumeClaim apiVersion: v1 metadata: name: mqtt-volumeclaim namespace: mqtt spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi EOF kubectl create -f /var/data/mqtt/mqtt-volumeclaim.yaml
Create nodeport service¶
I like to expose my services using nodeport (limited to ports 30000-32767), and then use an external haproxy load balancer to make these available externally. (This avoids having to pay per-port changes for a loadbalancer from the cloud provider)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
cat <<EOF > /var/data/mqtt/service-nodeport.yml kind: Service apiVersion: v1 metadata: name: mqtt-nodeport namespace: mqtt spec: selector: app: mqtt type: NodePort ports: - name: mqtts port: 8883 protocol: TCP nodePort : 30883 EOF kubectl create -f /var/data/mqtt/service-nodeport.yml
It's not always desirable to have sensitive data stored in your .yml files. Maybe you want to check your config into a git repository, or share it. Using Kubernetes Secrets means that you can create "secrets", and use these in your deployments by name, without exposing their contents.
1 2 3 4 5 6 7 8
echo -n "myapikeyissosecret" > cloudflare-key.secret echo -n "myemailaddress" > cloudflare-email.secret echo -n "myemailaddress" > letsencrypt-email.secret kubectl create secret -n mqtt generic mqtt-credentials \ --from-file=cloudflare-key.secret \ --from-file=cloudflare-email.secret \ --from-file=letsencrypt-email.secret
Because. See my blog post here for the pain of hunting invisible newlines, that's why!
Now that we have a volume, a service, and a namespace, we can create a deployment for the mqtt pod. Note below the use of volume mounts, environment variables, as well as the secrets.
I share (with my patreon patrons) a private "premix" git repository, which includes necessary .yml files for all published recipes. This means that patrons can launch any recipe with just a
git pull and a
kubectl create -f *.yml 👍
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74
cat <<EOF > /var/data/mqtt/mqtt.yml apiVersion: extensions/v1beta1 kind: Deployment metadata: namespace: mqtt name: mqtt labels: app: mqtt spec: replicas: 1 selector: matchLabels: app: mqtt template: metadata: labels: app: mqtt spec: containers: - image: funkypenguin/mqtt-certbot-dns imagePullPolicy: Always # only uncomment these to get the container to run so that we can transfer files into the PV # command: [ "/bin/sleep" ] # args: [ "1h" ] env: - name: DOMAIN value: "*.funkypenguin.co.nz" - name: EMAIL valueFrom: secretKeyRef: name: mqtt-credentials key: letsencrypt-email.secret - name: CLOUDFLARE_EMAIL valueFrom: secretKeyRef: name: mqtt-credentials key: cloudflare-email.secret - name: CLOUDFLARE_KEY valueFrom: secretKeyRef: name: mqtt-credentials key: cloudflare-key.secret # uncomment this to test LetsEncrypt validations # - name: TESTCERT # value: "true" name: mqtt resources: requests: memory: "50Mi" cpu: "0.1" volumeMounts: # We need the LE certs to persist across reboots to avoid getting rate-limited (bad, bad) - name: mqtt-volumeclaim mountPath: /etc/letsencrypt # A configmap for the mosquitto.conf file - name: mosquitto-conf mountPath: /mosquitto/conf/mosquitto.conf subPath: mosquitto.conf # A configmap for the mosquitto passwd file - name: mosquitto-passwd mountPath: /mosquitto/conf/passwd subPath: passwd volumes: - name: mqtt-volumeclaim persistentVolumeClaim: claimName: mqtt-volumeclaim - name: mosquitto-conf configMap: name: mosquitto.conf - name: mosquitto-passwd configMap: name: passwd EOF kubectl create -f /var/data/mqtt/mqtt.yml
Check that your deployment is running, with
kubectl get pods -n mqtt. After a minute or so, you should see a "Running" pod, as illustrated below:
1 2 3 4
[davidy:~/Documents/Personal/Projects/mqtt-k8s] 130 % kubectl get pods -n mqtt NAME READY STATUS RESTARTS AGE mqtt-65f4d96945-bjj44 1/1 Running 0 5m [davidy:~/Documents/Personal/Projects/mqtt-k8s] %
To actually use your new MQTT broker, you'll need to connect to any one of your nodes (
kubectl get nodes -o wide) on port 30883 (the nodeport service we created earlier). More info on that, and a loadbalancer design, to follow shortly :)
Chef's Notes 📓¶
Tip your waiter (support me) 👏¶
Did you receive excellent service? Want to make your waiter happy? (..and support development of current and future recipes!) See the support page for (free or paid) ways to say thank you! 👏
Flirt with waiter (subscribe) 💌¶
Want to know now when this recipe gets updated, or when future recipes are added? Subscribe to the RSS feed, or leave your email address below, and we'll keep you updated. (double-opt-in, no monkey business, no spam either - check the archive for proof!)