You can do WHAT with a Kafka proxy?

At Current 2026, I realized that nobody knows exactly what a Kafka proxy can do.

Most engineers and architects think it's just some kind of reverse-proxy for Kafka (think nginx) to do routing and used to bridge a legacy or non-native client to the cluster.

That's not it. It's barely the start of it.

Encryption

For instance, an engineer at a UK building society had a hard requirement: encrypt personally identifiable fields before they ever hit Kafka: emails, national insurance numbers, that kind of data.

His team built encryption into the application layer. Every producer that touched PII got encryption code. Every consumer got decryption code. Key handling, rotation, etc. to manage across services.

Something like this:

// In every producer that touches PII...
public ProducerRecord<String, Customer> encrypt(Customer c) {
    c.setEmail(crypto.encrypt(c.getEmail(), keyRef("pii-key")));
    c.setSsn(crypto.encrypt(c.getSsn(), keyRef("pii-key")));
    return new ProducerRecord<>("customers", c.getId(), c);
}

// And the mirror image in every consumer...
public Customer decrypt(Customer c) {
    c.setEmail(crypto.decrypt(c.getEmail()));
    c.setSsn(crypto.decrypt(c.getSsn()));
    return c;
}

Multiply that by all the micro-services to update and maintain now (cross languages, versioning, access to KMS etc.). That's quite expensive, at implementation time and to maintain.

He didn't know a Kafka proxy could have done the whole thing at the record level, outside the apps. When we chat about it, he just realized he might have done a mistake.

What a Kafka proxy does

A Kafka proxy sits between your clients and your brokers and speaks the Kafka protocol. Clients connect to it exactly like they'd connect to a broker. No SDK, no app changes. It works for Kafka clients, Kafka Connect, Kafka Streams, Flink, Spark, etc. It's fully transparent to them.

It makes it a natural place to put policy that doesn't belong inside your application and doesn't belong inside the cluster either.

Encryption is the obvious one. Instead of touching dozens of applications, you declare the rule once. With Conduktor Gateway it's what we call an interceptor: a small piece of config applied to traffic matching a topic pattern. Roughly:

{
  "kind": "Interceptor",
  "apiVersion": "gateway/v2",
  "metadata": { "name": "encrypt-customer-pii" },
  "spec": {
    "pluginClass": "io.conduktor.gateway.interceptor.EncryptionPlugin",
    "priority": 100,
    "config": {
      "topic": "customers.*",
      "kmsConfig": { "kms": "VAULT", "vault": { "uri": "https://vault:8200" } },
      "recordValue": {
        "fields": [
          { "fieldName": "email", "algorithm": "AES256_GCM", "keySecretId": "pii-key" },
          { "fieldName": "ssn",   "algorithm": "AES256_GCM", "keySecretId": "pii-key" }
        ]
      }
    }
  }
}

The proxy encrypts the fields on the way in and authorized consumers get them decrypted on the way out, everyone else gets ciphertext. The application code shrinks back to just... sending a record, not dealing with KMS and secrets:

// Same producer, after.
producer.send(new ProducerRecord<>("customers", c.getId(), c));

Because the rule lives in one declarative place (the proxy), you can do things that are painful at the app layer.

For instance, crypto-shredding for GDPR. Delete the key, and every message encrypted with it becomes unreadable, instantly, across all your retention. You don't go hunting through topics for one person's data. You revoke a key. Done.

Masking, validation, isolation: same one place

Once the proxy is in the Kafka path, the same pattern opens a lot of doors:

Field-level masking: show j***@example.com to one team, the real value to another, same topic.
Schema and payload validation: reject malformed records at the edge instead of poisoning a downstream consumer.
Topic aliasing for migration: point clients at a stable name while you move the real topic between clusters.
Virtual clusters: carve one physical cluster into isolated tenants without standing up new infrastructure.
Audit and policy enforcement: log and gate access without patching the broker or the client.

None of that touches application code or broker config. It's policy, declared once, enforced in the path.

Why this connects to cost and self-service

The other big topic at the conference was cost. Conduktor published a field guide on where Kafka costs hide in April.

Then the self-service conversation: teams want developers to create topics and request access in autonomy, but simple Topic on GitOps solution is just not enough, because self-service without guardrails easily turns Kafka into a mess:

"If somebody goes onto the tool and adds in something ridiculous, like a thousand partitions, we need someone to have eyes on that. That's something we've learned we can't let go of."

Look at what encryption-at-the-proxy, cost guardrails, and self-service approval gates have in common. They're all policy that belongs between your developers and your brokers, not baked into either one. Push it into the app and you copy-paste it dozens of time. Push it into the cluster and you can't change it without a migration. Put it in the layer in between, declare it once, and you can actually govern it.

To build AI agents on top of streaming, the plumbing must come first: ownership, schema discipline, key custody, data quality before the data even enters Kafka, etc. A proxy that enforces structure and policy is a big chunk of that plumbing.

So where does your policy live?

What people think a proxy does is pass packets. What it really does is more of a safekeeper that holds the policy.

If you've got encryption, masking, validation, or multi-tenant isolation scattered across your services right now, it's worth asking whether any of it should be living one layer down instead.

Want to go deeper? The Gateway overview walks through the interceptor model, and the original Current 2026 write-up has the rest of what we heard on the floor.

推荐订阅源