MetalLB¶

MetalLB offers a network load balancer implementation which workes on "bare metal" (as opposed to a cloud provider).

MetalLB does two jobs:

1. Provides address allocation to services out of a pool of addresses which you define
2. Announces these addresses to devices outside the cluster, either using ARP/NDP (L2) or BGP (L3)

Preparation¶

Allocations¶

You'll need to make some decisions re IP allocations.

• What is the range of addresses you want to use for your LoadBalancer service pool? If you're using BGP, this can be a dedicated subnet (i.e. a /24), and if you're not, this should be a range of IPs in your existing network space for your cluster nodes (i.e., 192.168.1.100-200)
• If you're using BGP, pick two private AS numbers between 64512 and 65534 inclusively.

Namespace¶

We need a namespace to deploy our HelmRelease and associated ConfigMaps into. Per the flux design, I create this in my flux repo at flux-system/namespaces/namespace-metallb.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: metallb-system

HelmRepository¶

Next, we need to define a HelmRepository (a repository of helm charts), to which we'll refer when we create the HelmRelease. We only need to do this once per-repository. In this case, we're using the (prolific) bitnami chart repository, so per the flux design, I create this in my flux repo at flux-system/helmrepositories/helmrepository-bitnami.yaml:

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
  name: bitnami
  namespace: flux-system
spec:
  interval: 15m
  url: https://charts.bitnami.com/bitnami

Kustomization¶

Now that the "global" elements of this deployment (Namespace and HelmRepository) have been defined, we do some "flux-ception", and go one layer deeper, adding another Kustomization, telling flux to deploy any YAMLs found in the repo at /metallb-system. I create this Kustomization in my flux repo at flux-system/kustomizations/kustomization-metallb.yaml:

apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
kind: Kustomization
metadata:
  name: metallb--metallb-system
  namespace: flux-system
spec:
  interval: 15m
  path: ./metallb-system
  prune: true # remove any elements later removed from the above path
  timeout: 2m # if not set, this defaults to interval duration, which is 1h
  sourceRef:
    kind: GitRepository
    name: flux-system
  validation: server
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: metallb-controller
      namespace: metallb-system

ConfigMap (for HelmRelease)¶

Now we're into the metallb-specific YAMLs. First, we create a ConfigMap, containing the entire contents of the helm chart's values.yaml. Paste the values into a values.yaml key as illustrated below, indented 4 tabs (since they're "encapsulated" within the ConfigMap YAML). I create this in my flux repo at metallb/configmap-metallb-helm-chart-value-overrides.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: metallb-helm-chart-value-overrides
  namespace: metallb-system
data:
  values.yaml: |-
    ## @section Global parameters
    ## Global Docker image parameters
    ## Please, note that this will override the image parameters, including dependencies, configured to use the global value
    ## Current available global Docker image parameters: imageRegistry, imagePullSecrets and storageClass

    ## @param global.imageRegistry Global Docker image registry
    ## @param global.imagePullSecrets Global Docker registry secret names as an array
    ##
    global:
    imageRegistry: ""
    ## E.g.
    ## imagePullSecrets:
    ##   - myRegistryKeySecretName
    <snip>
    prometheus:
        ## Prometheus Operator service monitors
        ##
        serviceMonitor:
        ## @param speaker.prometheus.serviceMonitor.enabled Enable support for Prometheus Operator
        ##
        enabled: false
        ## @param speaker.prometheus.serviceMonitor.jobLabel Job label for scrape target
        ##
        jobLabel: "app.kubernetes.io/name"
        ## @param speaker.prometheus.serviceMonitor.interval Scrape interval. If not set, the Prometheus default scrape interval is used
        ##
        interval: ""
        ## @param speaker.prometheus.serviceMonitor.metricRelabelings Specify additional relabeling of metrics
        ##
        metricRelabelings: []
        ## @param speaker.prometheus.serviceMonitor.relabelings Specify general relabeling
        ##
        relabelings: []

Then work your way through the values you pasted, and change any which are specific to your configuration. I'd recommend changing the following:

• existingConfigMap: metallb-config: I prefer to set my MetalLB config independently of the chart config, so I set this to metallb-config, which I then define below.
• commonAnnotations: Anticipating the future use of Reloader to bounce applications when their config changes, I add the configmap.reloader.stakater.com/reload: "metallb-config" annotation to all deployed objects, which will instruct Reloader to bounce the daemonset if the ConfigMap changes.

ConfigMap (for MetalLB)¶

Finally, it's time to actually configure MetalLB! As discussed above, I prefer to configure the helm chart to apply config from an existing ConfigMap, so that I isolate my application configuration from my chart configuration (and make tracking changes easier). In my setup, I'm using BGP against a pair of pfsense¹ firewalls, so per the official docs, I use the following configuration, saved in my flux repo as flux-system/configmap-metallb-config.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: metallb-config
data:
  config: |
    peers:
    - peer-address: 192.168.33.2  
      peer-asn: 64501
      my-asn: 64500
    - peer-address: 192.168.33.4  
      peer-asn: 64501
      my-asn: 64500

    address-pools:
    - name: default
      protocol: bgp
      avoid-buggy-ips: true
      addresses:
      - 192.168.32.0/24

apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: metallb-config
data:
  config: |
    address-pools:
    - name: default
    protocol: layer2
    addresses:
    - 192.168.1.240-192.168.1.250

HelmRelease¶

Lastly, having set the scene above, we define the HelmRelease which will actually deploy MetalLB into the cluster, with the config and extra ConfigMap we defined above. I save this in my flux repo as metallb/helmrelease-metallb.yaml:

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: metallb
  namespace: metallb-system
spec:
  chart:
    spec:
      chart: metallb
      version: 2.x
      sourceRef:
        kind: HelmRepository
        name: bitnami
        namespace: flux-system
  interval: 15m
  timeout: 5m
  releaseName: metallb
  valuesFrom:
  - kind: ConfigMap
    name: metallb-helm-chart-value-overrides
    valuesKey: values.yaml # This is the default, but best to be explicit for clarity

Deploy MetalLB¶

Having committed the above to your flux repository, you should shortly see a metallb kustomization, and in the metallb-system namespace, a controller and a speaker pod for every node:

root@cn1:~# kubectl get pods -n metallb-system -o wide
NAME                                  READY   STATUS    RESTARTS   AGE   IP              NODE   NOMINATED NODE   READINESS GATES
metallb-controller-779d8686f6-mgb4s   1/1     Running   0          21d   10.0.6.19       wn3    <none>           <none>
metallb-speaker-2qh2d                 1/1     Running   0          21d   192.168.33.24   wn4    <none>           <none>
metallb-speaker-7rz24                 1/1     Running   0          21d   192.168.33.22   wn2    <none>           <none>
metallb-speaker-gbm5r                 1/1     Running   0          21d   192.168.33.23   wn3    <none>           <none>
metallb-speaker-gzgd2                 1/1     Running   0          21d   192.168.33.21   wn1    <none>           <none>
metallb-speaker-nz6kd                 1/1     Running   0          21d   192.168.33.25   wn5    <none>           <none>
root@cn1:~#

- key: "node-role.kubernetes.io/master"
  effect: "NoSchedule"

How do I know it's working?¶

If you used my template repository to start off your flux deployment strategy, then the podinfo helm chart has already been deployed. By default, the podinfo service is in ClusterIP mode, so it's only reachable within the cluster.

Edit your podinfo helmrelease configmap (/podinfo/configmap-podinfo-helm-chart-value-overrides.yaml), and change this:

    <snip>
    # Kubernetes Service settings
    service:
      enabled: true
      annotations: {}
      type: ClusterIP
    <snip>

To:

    <snip>
    # Kubernetes Service settings
    service:
      enabled: true
      annotations: {}
      type: LoadBalancer
    <snip>

Commit your changes, wait for a reconciliation, and run kubectl get services -n podinfo. All going well, you should see that the service now has an IP assigned from the pool you chose for MetalLB!

Chef's notes 📓¶

Tip your waiter (sponsor) 👏¶

Did you receive excellent service? Want to compliment the chef? (..and support development of current and future recipes!) Sponsor me on Github / Patreon, or see the contribute page for more (free or paid) ways to say thank you! 👏

Employ your chef (engage) 🤝¶

Is this too much of a geeky PITA? Do you just want results, stat? I do this for a living - I provide consulting and engineering expertise to businesses needing short-term, short-notice support in the cloud-native space, including AWS/Azure/GKE, Kubernetes, CI/CD and automation.

Learn more about working with me here.

Flirt with waiter (subscribe) 💌¶

Want to know now when this recipe gets updated, or when future recipes are added? Subscribe to the RSS feed, or leave your email address below, and we'll keep you updated.

Your comments? 💬¶