Following up on the previous post Valkey: running in Kubernetes – Helm, monitoring, ACL – we set up Valkey/Redis there, now it’s time to add MongoDB.

This is for our new service, which is still in the experimental/PoC phase, but will most likely go to production, so the setup is a bit “under-production” – we’re building it with the idea that later we’ll properly manage and monitor all of this, but for now we’re not too worried about security tasks.

Just like with Valkey, we’ll run MongoDB in Kubernetes, but unlike Valkey, this one is a bit more complicated and a bit more painful, because:

• there’s an enterprise version of MongoDB, and part of the components and documentation are geared toward it, which is a bit confusing – because there’s a lot of documentation
• there’s no decent and simple Helm chart – because the system itself is a bit more complex than Redis (well, unless you’re deploying Redis Cluster)
  • there’s a Bitnami chart – but we’re not our own enemies 🙂

So for running it in Kubernetes, I went with the MCK approach – MongoDB Controllers for Kubernetes (MCK).

So, here’s the plan:

• install the MongoDB Operator itself in its own Kubernetes Namespace
• create a test MongoDB Replica Set “cluster” in a separate Namespace – but with a single node

Since we’ll be using a Kubernetes Operator and CRD for MongoDB, I’d recommend reading up on how this works “under the hood” – Kubernetes: Kubernetes API, API Groups, CRD and etcd and Kubernetes: what is a Kubernetes Operator and CustomResourceDefinition.

Install MongoDB Operator

All the values are nicely documented in MongoDB Controllers for Kubernetes Operator Helm Installation Settings.

All the default values can be found in the repo mongodb/helm-charts/main/charts/mongodb-kubernetes/values.yaml.

It might also be worth checking out the mongodb-kubernetes.yaml manifest – basically all the resources that the operator’s own Helm chart creates.

For now we’re installing the operator by hand into a test Kubernetes Namespace – we’ll move it into automation later.

Adding the repo:

$ helm repo add mongodb https://mongodb.github.io/helm-charts

Writing a simple values file:

operator:

  env: dev
  
  watchNamespace: test-mongodb-service-ns

  watchedResources:
    - mongodbcommunity

  replicas: 1

  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 500m
      memory: 512Mi

  createOperatorServiceAccount: true
  createResourcesServiceAccountsAndRoles: true

  enableClusterMongoDBRoles: false
  enablePVCResize: true

Here:

• env: the log level is set based on the value in env
• watchNamespace: we specify the Kubernetes Namespace where MongoDBCommunity will later live – the operator creates Roles && RoleBindings here, and it will only watch its Custom Resources in these NS
• watchedResources: we’ll only be using the MongoDBCommunity CRD, without Enterprise, Ops Manager, Search, etc. – so we limit the resources here
• enableClusterMongoDBRoles: we’re not using the ClusterMongoDBRole CRD, roles for MongoDB will be set at the MongoDBCommunity level
• enablePVCResize: enables the ability to resize PVCs (the StorageClass needs to have allowVolumeExpansion)
• resources: adding this right away, we’ll tune the values for production later

The operator’s Helm chart will install the necessary CRDs, the operator’s own ClusterRole and RoleBinding in its own namespace – and a RoleBinding plus the required ServiceAccounts in the namespace of our test MongoDB instance.

Creating the namespaces:

$ kk create ns test-mongodb-operator-ns

$ kk create ns test-mongodb-service-ns

Deploying the operator:

$ helm upgrade --install mongodb-kubernetes-operator \
  mongodb/mongodb-kubernetes \
  --namespace test-mongodb-operator-ns \
  --create-namespace \
  -f operator-values.yaml

Checking the pods:

$ kk get pod
NAME                                           READY   STATUS    RESTARTS   AGE
mongodb-kubernetes-operator-6b4cb5955b-wk9q5   1/1     Running   0          28s

To create MongoDB instances in the watchNamespace, the operator creates a ServiceAccount there – let’s check that it exists and that this ServiceAccount has permissions to create a Kubernetes StatefulSet:

$ kk auth can-i create statefulsets.apps \
  --namespace test-mongodb-service-ns \
  --as system:serviceaccount:test-mongodb-operator-ns:mongodb-kubernetes-operator
yes

Checking the MongoDBCommunity CustomResourceDefinition:

$ kk get crd mongodbcommunity.mongodbcommunity.mongodb.com
NAME                                            CREATED AT
mongodbcommunity.mongodbcommunity.mongodb.com   2026-07-22T13:36:50Z

By default the operator sends MongoDB product telemetry – you can disable it with:

operator:
  telemetry:
    enabled: false

The operator is ready – now we can launch MongoDB itself.

Deploying the MongoDB Community instance

You can check out the CRD itself here – mongodbcommunity.mongodb.com_mongodbcommunity.yaml.

With it we’re actually describing our own MongoDB, so this we do in a different namespace, test-mongodb-service-ns, where our own service that will be using this MongoDB will live.

For now we’re creating Kubernetes Secrets and passwords by hand, later we’ll do it properly – with AWS Secrets Manager and the External Secrets Operator.

Creating a secret with the root password for this MongoDB instance:

$ kk -n test-mongodb-service-ns create secret generic agents-mainframe-mongodb-admin-password \
  --from-literal=password='test-mongodb-admin-password'

Creating the mongodb-community.yaml manifest:

apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
  name: agents-mainframe-mongodb
spec:
  type: ReplicaSet
  members: 1
  version: "8.0.13"

  security:
    authentication:
      modes:
        - SCRAM

  users:
    - name: admin
      db: admin
      passwordSecretRef:
        name: agents-mainframe-mongodb-admin-password
      scramCredentialsSecretName: agents-mainframe-mongodb-admin-scram
      roles:
        - name: root
          db: admin

Here:

• members: the number of MongoDB processes in the MongoDB Replica Set (not to be confused with a Kubernetes ReplicaSet)
• authentication.modes: SCRAM – standard username/password authentication
• users: this is where we describe RBAC – setting the name of the root user
  • in passwordSecretRef we pass the name of the Kubernetes Secret storing its password (the one we created above)
  • scramCredentialsSecretName: the name of the Kubernetes Secret for the MongoDB Operator, see below
  • in roles – we set the value root, all the roles are listed in the Built-In Roles documentation

Deploying:

$ kk -n test-mongodb-service-ns apply -f mongodb-community.yaml
mongodbcommunity.mongodbcommunity.mongodb.com/agents-mainframe-mongodb created

Checking the Pods:

$ kk -n test-mongodb-service-ns get pod
NAME                         READY   STATUS    RESTARTS   AGE
agents-mainframe-mongodb-0   2/2     Running   0          2m30s

Checking the state of the CustomResource:

$ kk -n test-mongodb-service-ns get mongodbcommunity agents-mainframe-mongodb
NAME                       PHASE     VERSION
agents-mainframe-mongodb   Running   8.0.13

Checking authentication:

$ kk -n test-mongodb-service-ns exec -it \
  agents-mainframe-mongodb-0 \
  -c mongod \
  -- mongosh \
  --username admin \
  --password \
  --authenticationDatabase admin
Enter password: ***************************
Warning: Could not access file: EACCES: permission denied, mkdir '/data/db/.mongodb'
Current Mongosh Log ID: 6a60ce6f6671cfc48fce5f46
Connecting to:          mongodb://<credentials>@127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&authSource=admin&appName=mongosh+2.5.8

...
Error: Could not open history file.
REPL session history will not be persisted.

...

agents-mainframe-mongodb [direct: primary] test> 

There’s an error in the logs, “EACCES: permission denied, mkdir ‘/data/db/.mongodb’” – but this is from the MongoDB Shell, not the mongod daemon itself: right after it there’s also “Error: Could not open history file.”

So we ignore it.

And through db.runCommand() let’s try running some command, for example – connectionStatus:

agents-mainframe-mongodb [direct: primary] test> db.runCommand({ connectionStatus: 1 })
{
  authInfo: {
    authenticatedUsers: [ { user: 'admin', db: 'admin' } ],
    authenticatedUserRoles: [ { role: 'root', db: 'admin' } ]
  },
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1784729214, i: 1 }),
    signature: {
      hash: Binary.createFromBase64('cVOqP4fgjvw/kg8m6SnhJ03AB1A=', 0),
      keyId: Long('7665352957805723654')
    }
  },
  operationTime: Timestamp({ t: 1784729214, i: 1 })
}

Everything works.

Another interesting and useful thing to know about – the Kubernetes Secrets that get created by the MongoDB Operator itself when we deploy a MongoDBCommunity resource:

$ kk -n test-mongodb-service-ns get secret
NAME                                                     TYPE     DATA   AGE
agents-mainframe-mongodb-admin-admin                     Opaque   4      19m
agents-mainframe-mongodb-admin-password                  Opaque   1      26m
agents-mainframe-mongodb-admin-scram-scram-credentials   Opaque   6      21m
agents-mainframe-mongodb-agent-password                  Opaque   1      21m
agents-mainframe-mongodb-config                          Opaque   1      21m
agents-mainframe-mongodb-keyfile                         Opaque   1      21m

Here, agents-mainframe-mongodb-admin-password is the one we created by hand and passed in the CR manifest in the passwordSecretRef field, and the others are:

• agents-mainframe-mongodb-admin-scram-scram-credentials: this is a Secret for the operator itself – salt, sha keys
  • the name is generated from the value in scramCredentialsSecretName in our CR manifest, to which the operator appends the suffix “-scram-credentials“
• agents-mainframe-mongodb-admin-admin: here the operator generates a connection string for connecting, in the format mongodb://<username>:<password>@<host>:<port>/<database>?<options>"
  • the name is generated from <MongoDBCommunity name>-<auth database>-<username>
  • it contains two connection strings – connectionString.standard and connectionString.standardSrv: the second one uses Kubernetes DNS instead of pod names – we use that one
• agents-mainframe-mongodb-agent-password: the internal MongoDB Agent password, created by the operator – the agent uses this password to manage mongod on this MongoDB instance, not something we care about
• agents-mainframe-mongodb-config: automation configuration for the MongoDB Agent – the desired state of the deployment, replica set, users, and other parameters – updated by the operator whenever the MongoDBCommunity changes, we don’t really care about this one either

And that’s pretty much it.

We can start getting ready for Dev/Staging/Prod:

• add podAntiAffinity, topologySpreadConstraints (see Kubernetes: Pods and WorkerNodes – controlling pod placement on nodes)
• enable a PodDisruptionBudget for environments with replicas > 1
• set up PVC with retention policy == Retain
• add resources.requests/limits separately for the mongod and mongodb-agent containers
• set up backups
• set up monitoring (there’s an example, mongodb-prometheus-sample.yaml)

As for backups – plain EBS snapshots aren’t enough/reliable, options are:

• mongodump/mongorestore + AWS S3 (see The MongoDB Database Tools Documentation)
• EBS snapshots – but on a separate MongoDB instance, running db.fsyncLock() and db.fsyncUnlock() (see Back Up a Self-Managed Deployment with Filesystem Snapshots)
• or set up Percona Backup for MongoDB