Skip to content

Authenticate to Kubernetes with Keycloak OIDC on K3s

This recipe describes how to configure K3s for OIDC authentication against a Keycloak instance.

For details on why you'd want to do this, see the Kubernetes Authentication Guide.

Requirements

Ingredients

Setup K3s for OIDC auth

If you followed the k3s install guide, you'll have installed K3s with a command something like this:

MYSECRET=iambatman
curl -fL https://get.k3s.io | K3S_TOKEN=${MYSECRET} \
    sh -s - --disable traefik server

To configure the apiserver to perform OIDC authentication, you need to add some extra kube-apiserver arguments. There are two ways to do this:

  1. Append the arguments to your curl | bash command, like a lunatic
  2. Add the arguments to a config file which K3s will parse upon start, like a gentleman

Here's the lunatic option:

Lunatic curl | bash option
--kube-apiserver-arg=oidc-issuer-url=https://keycloak.example.com/auth/realms/master/
--kube-apiserver-arg=oidc-client-id=kube-apiserver
--kube-apiserver-arg=oidc-username-claim=email
--kube-apiserver-arg=oidc-groups-claim=groups

And here's the gentlemanly option:

Create /etc/rancher/k3s/config.yaml, and add:

Gentlemanly YAML config option
kube-apiserver-arg:
- "oidc-issuer-url=https://keycloak.example.com/auth/realms/master/"
- "oidc-client-id=kube-apiserver"
- "oidc-username-claim=email"
- "oidc-groups-claim=groups"

Now restart k3s (systemctl restart k3s on Ubuntu), and confirm it starts correctly by watching the logs (journalctl -u k3s -f on Ubuntu)

Assuming nothing explodes, you're good-to-go on attempting to actually connect...

Install kubelogin

For CLI-based access to your cluster, you'll need a "helper" to perform the OIDC magic on behalf of kubectl. Install int128/kubelogin, which is design suited to this purpose.

Use kubelogin to test your OIDC parameters, by running:

kubectl oidc-login setup \
  --oidc-issuer-url=ISSUER_URL \
  --oidc-client-id=YOUR_CLIENT_ID \
  --oidc-client-secret=YOUR_CLIENT_SECRET

All going well, your browser will open a new window, logging you into Keycloak, and on the CLI you should get output something like this:

~ ❯ kubectl oidc-login setup --oidc-issuer-url=https://keycloak.example.com/auth/realms/master/ --oidc-client-id=kube-apiserver --oidc-client-secret=<your secret>
authentication in progress...

## 2. Verify authentication

You got a token with the following claims:

{
  "exp": 1700008379,
  "iat": 1700007479,
  "auth_time": 1700006251,
  "jti": "80760d79-3404-406c-bfd9-5c41783b0a5a",
  "iss": "https://keycloak.example/auth/realms/master",
  "aud": "kube-apiserver",
  "sub": "a612b1ae-63e1-4698-bcc8-9ba8b8b7fb84",
  "typ": "ID",
  "azp": "kube-apiserver",
  "nonce": "fwXjVCFM6xosn9yctYEducYBdy4KcnOqbaDxHPRWsTg",
  "session_state": "851804e4-e479-46ac-93b4-c89ac37aa7a3",
  "at_hash": "7lv4QY3h54maW6S5E--kgg",
  "acr": "0",
  "sid": "851804e4-e479-46ac-93b4-c89ac37aa7a3",
  "email_verified": false,
  "name": "David Young",
  "groups": [
    "admin-kiali",
    "admin-harbor",
    "admin-graylog",
    "admin-kubernetes",
    "concourse-main",
    "admin-keycloak",
    "admin-grafana"
  ],
  "preferred_username": "davidy",
  "given_name": "David",
  "family_name": "Young",
  "email": "davidy@funkypenguin.co.nz"
}

Huzzah, authentication works! 🥳

Tip

Make sure you see a groups claim in the output above, and if you don't, revisit your client's mapper settings as when you configured as an OIDC provider for kube-apiserver.

Assemble your kubeconfig

Your kubectl access to k3s uses a kubeconfig file at /etc/rancher/k3s/k3s.yaml. Treat this file as a root password - it's includes a long-lived token which gives you clusteradmin ("god mode" on your cluster.)

Copy the k3s.yaml file to your local desktop (the one with a web browser), into $HOME/.kube/config, and modify it, changing server: https://127.0.0.1:6443 to match the URL of (one of) your control-plane node(s).

Test using kubectl cluster-info locally, ensuring that you have access.

Amend the kubeconfig file for your OIDC user, by running a variation of:

kubectl config set-credentials oidc \
 --exec-api-version=client.authentication.k8s.io/v1beta1 \
 --exec-command=kubectl \
 --exec-arg=oidc-login \
 --exec-arg=get-token \
 --exec-arg=--oidc-issuer-url=https://keycloak.example.com/auth/realms/master/ \
 --exec-arg=--oidc-client-id=kube-apiserver \
 --exec-arg=--oidc-client-secret=<your client secret>

Test your OIDC powerz by running kubectl --user=oidc cluster-info.

Now gasp in dismay as you discover that your request was denied for lack of access! 😱

Error from server (Forbidden): services is forbidden: User "oidc:davidy@funkypenguin.co.nz" 
cannot list resource "services" in API group "" in the namespace "kube-system"

Create clusterrolebinding

That's what you wanted, right? Security? Locking out unauthorized users? Ha.

Now that we've confirmed that kube-apiserver knows your identity (authn), create a clusterrolebinding to tell it what your identity is authorized to do (authz), based on your group membership.

The following is a simple clusterrolebinding which will grant all members of the admin-kube-apiserver full access (cluster-admin), to get you started:

/authentic/clusterrolebinding-oidc-group-admin-kube-apiserver.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: oidc-group-admin-kube-apiserver
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin # (1)!
subjects:
- kind: Group
  name: oidc:admin-kube-apiserver # (2)!
  1. The role to bind
  2. The subject (group, in this case) of the binding

Apply your clusterrolebinding using the usual GitOps magic (I put mine in /authentic/clusterrolebinding-oidc-group-admin-kube-apiserver.yaml).

Run kubectl --user=oidc cluster-info again, and confirm you are now authorized to see the cluster details.

If this works, set your user context permanently, using kubectl config set-context --current --user=oidc.

whoami?

Run kubectl krew install whoami to install the whoami plugin, and then kubectl whoami to confirm you're logged in with your OIDC account

You now have OIDC-secured CLI access to your cluster!

Summary

What have we achieved?

We've setup our K3s cluster to authenticate against Keycloak, running on that same cluster! We can now create multiple users (with multiple levels of access) without having to provide them with more access than they need, and we can deploy kube-apiserver-integrated tools like Kubernetes Dashboard or Weaveworks GitOps for nice secured UIs.

Summary

Created:

  • K3s cluster with OIDC authentication against Keycloak
  • Ability to support:
  • We've also retained our static, K3s-generated kubernetes-admin credentials in case OIDC auth fails at some point (keep them safe!)

What's next?

Deploy Weave GitOps to visualize your Flux / GitOps state, and Kubernetes Dashboard for UI management of your cluster!

Chef's notes 📓

  1. Later on, as we add more applications which need kube-apiserver authentication, we'll add more redirect URIs. ↩

Tip your waiter (sponsor) 👏

Did you receive excellent service? Want to compliment the chef? (..and support development of current and future recipes!) Sponsor me on Github / Ko-Fi / Patreon, or see the contribute page for more (free or paid) ways to say thank you! 👏

Employ your chef (engage) 🤝

Is this too much of a geeky PITA? Do you just want results, stat? I do this for a living - I'm a full-time Kubernetes contractor, providing consulting and engineering expertise to businesses needing short-term, short-notice support in the cloud-native space, including AWS/Azure/GKE, Kubernetes, CI/CD and automation.

Learn more about working with me here.

Flirt with waiter (subscribe) 💌

Want to know now when this recipe gets updated, or when future recipes are added? Subscribe to the RSS feed, or leave your email address below, and we'll keep you updated.

Your comments? 💬