In this article we’ll explore how to use Traefik in Kubernetes combined with Cert-manager as an ACME (Automatic Certificate Management Environment) client to issue certificates through Let’s Encrypt.

If instead of Kubernetes you’re running docker-compose, Major Hayden has an excellent tutorial on how to configure Wildcard LetsEncrypt certificates with Traefik and Cloudflare.

In order to issue wildcard certificates we need to prove to a Certificate Authority (CA) that we own the domain. One way to prove ownership is with a DNS-01 challenge.

We’ll be using Cloudflare as a DNS provider, but the examples should be easily adapted for other providers.

Overview#

There’s a lot of moving parts when dealing with TLS/SSL certificates and public key infrastructure which we’ll not cover here.

Below is a simplified overview of the certificate issuance flow we’ll implement. If you feel the explanation is lacking or just plain wrong I’d be happy to hear from you!

  
flowchart RL
  
subgraph LE[Let's Encrypt]
  CA
end
subgraph Cloudflare
  TXT
end
subgraph Issuer
  Token[API Token]
end
subgraph Certificate
  TLS[TLS Secret]
end
subgraph Traefik
  Ingress
end

Issuer --> LE
Token --> TXT
LE --> |DNS-01|Cloudflare
CA --> Issuer
Certificate --> Issuer
Traefik --> TLS

1. The Issuer asks Let’s Encrypt for a DNS-01 challenge.
2. Using the Cloudflare API Token the Issuer creates a TXT DNS record as an answer to the DNS-01 challenge.
3. Let’s Encrypt verifies the TXT record to establish trust.
4. After verification Let’s Encrypt will now sign certificate requests from the Issuer.
5. The Certificate resource asks the Issuer for a valid TLS certificate and stores it as a Secret.
6. Traefik attaches the TLS Secret to eligible Ingress and IngressRoute resources.

Traefik Proxy#

In their own words,

Traefik is an open-source Edge Router that makes publishing your services a fun and easy experience.

On Kubernetes Traefik can be installed using their Helm Chart

helm repo add traefik https://traefik.github.io/charts
helm repo update
helm install traefik traefik/traefik

I’ve described a different approach suitable for Argo CD and Kustomize in the Summary section.

Cert-manager#

While Traefik has native Let’s Encrypt integration, running multiple instances can raise some issues. To solve those issues we can either pay for Traefik Enterprise, or delegate the certificate lifecycle management to a centralised service, e.g. Cert-manager.

Cert-manager keeps a list of supported DNS-01 providers on their webpage.¹ Luckily for us Cloudflare is one of those providers, though any of the providers listed by Let’s Encrypt should be possible to integrate with.

We can install Cert-manager by using their Helm Chart

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --version v1.13.3 --set installCRDs=true

For other options check the Summary section or Cert-manager’s docs.

DNS Provider (Cloudflare)#

I went with Cloudflare as a DNS provider since they’re well known and offer a free tier. Creating an account and setting up your site to use Cloudflare is outside the scope of this article, but it should be pretty straightforward by going to their site.

Cert-manager has great documentation on how to configure DNS-01 providers. For Cloudflare we only need an API-token with the appropriate permissions.

After logging in to Cloudflare navigate to My Profile in the upper right corner, find the API Tokens pane in the left sidebar, and locate the Create Token button under User API Tokens.

Create a custom token and give it the following permissions

• Zone – Zone – Read
• Zone – DNS – Edit

For the Zones Resources it’s recommended to include all zones.

Take note of the token as we need it in the next section.

Configuration#

With a Cloudflare API token in hand, and having set up both Traefik and Cert-manager, we’re finally ready to configure wildcard certificates for use by Traefik!

First we create a secret² with the API token we got from Cloudflare. Using stringData instead of data allows us to use clear text content instead of having to base64 encode the token.³

1
2
3
4
5
6
7
8

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token
  namespace: traefik
type: Opaque
stringData:
  api-token: "<--CLOUDFLARE API TOKEN-->"

Then we create an Issuer resource providing the domain owner e-mail on line 9 and reference the API token secret for apiTokenSecretRef (line 16).

For testing purposes it might be a good idea to use Let’s Encrypt staging server instead of the production server to avoid being rate limited. To do so change the server on line 8 to https://acme-staging-v02.api.letsencrypt.org/directory.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: cloudflare-issuer
  namespace: traefik
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: "<--YOUR EMAIL-->"
    privateKeySecretRef:
      name: letsencrypt-key
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token

Next we create a Certificate which references the above Issuer by name on line 12, inserting your own domain name on line 4, 7, 9, and 10 where appropriate.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: wildcard-<--YOUR DOMAIN-->
  namespace: traefik
spec:
  secretName: wildcard-<--YOUR DOMAIN-->-tls
  dnsNames:
    - "<--YOUR DOMAIN-->"
    - "*.<--YOUR DOMAIN-->"
  issuerRef:
    name: cloudflare-issuer
    kind: Issuer

Verify that everything is ready by running

$ kubectl get certificate -n traefik

NAME             READY   SECRET               AGE
wildcard-<...>   True    wildcard-<...>-tls   5m

If the certificate is not ready, the logs of the Cert-manager pod can give you more information.

Default certificate#

Assuming everything is running fine you can start using the certificate by adding

1
2
3
4

tlsStore:
  default:
    defaultCertificate:
      secretName: wildcard-<--YOUR DOMAIN-->-tls

to the Traefik Helm Chart values.yaml.

This instructs Traefik to present the TLS Secret generated by the Certificate resource by default for all Ingress and IngressRoute resources.

Update Traefik to apply the new settings

helm upgrade traefik traefik/traefik --values values.yaml

and you should be all set!

Per Ingress#

It’s also possible to issue certificates per Ingress or IngressRoute resource. For instructions on how to do this you can read this blog post by Traefik where they use Cert-manager annotations to request certificates.

Summary#

I’m running Argo CD with Kustomize + Helm in an attempt to follow GitOps best-practices.

I’m employing an App of Apps Pattern where I’m using Argo CD’s ApplicationSet to generate Application resources for each folder. You can find my current homelab configuration on GitHub as a demonstration.

Cert-manager#

#cert-manager/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ns.yaml

helmCharts:
  - name: cert-manager
    repo: https://charts.jetstack.io
    version: 1.13.3
    releaseName: cert-manager
    namespace: cert-manager
    valuesInline:
      crds.enabled: true

#cert-manager/ns.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager

Traefik#

#traefik/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ns.yaml
  - cloudflare-token-cert-manager.yaml
  - cloudflare-issuer.yaml
  - cloudflare-cert.yaml

helmCharts:
  - name: traefik
    repo: https://traefik.github.io/charts
    version: 26.0.0
    releaseName: traefik
    namespace: traefik
    includeCRDs: true
    valuesFile: values.yaml

#traefik/values.yaml
tlsStore:
  default:
    defaultCertificate:
      secretName: wildcard-<--YOUR DOMAIN-->-tls

#traefik/ns.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: traefik

This should preferably be a SealedSecret² instead of an unencrypted Secret

#traefik/cloudflare-token-cert-manager.yaml
apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token
  namespace: traefik
type: Opaque
stringData:
  api-token: "<--CLOUDFLARE API TOKEN-->"

#traefik/cloudflare-issuer.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: cloudflare-issuer
  namespace: traefik
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: "<--YOUR EMAIL-->"
    privateKeySecretRef:
      name: letsencrypt-key
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token

#traefik/cloudflare-cert.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: wildcard-<--YOUR DOMAIN-->
  namespace: traefik
spec:
  secretName: wildcard-<--YOUR DOMAIN-->-tls
  dnsNames:
    - "<--YOUR DOMAIN-->"
    - "*.<--YOUR DOMAIN-->"
  issuerRef:
    name: cloudflare-issuer
    kind: Issuer