By Flavio Copes

How to use Cloudflare Queues to do work after the response is sent. Producers, consumers, batching, retries, and dead letter queues.

~~~

Some work shouldn’t happen while the user waits.

Think about a sign-up. You create the account, then you want to send a welcome email, update analytics, and ping Slack. The user doesn’t care about any of that. They just want to be logged in.

Cloudflare Queues let you say “do this later.” You drop a message on a queue, respond to the user immediately, and a separate handler processes the message in the background.

How a queue works

There are two sides:

• A producer puts messages on the queue. That’s your normal Worker.
• A consumer takes messages off and processes them. That’s a handler that runs on its own.

Cloudflare sits in the middle, holding the messages and delivering them reliably.

Set it up

Create the queue:

npx wrangler queues create my-app-events

Then wire up both sides in wrangler.jsonc. The producer gets a binding, the consumer points at the same queue:

{
  "queues": {
    "producers": [
      { "binding": "EVENTS", "queue": "my-app-events" }
    ],
    "consumers": [
      {
        "queue": "my-app-events",
        "max_batch_size": 10,
        "max_retries": 3,
        "dead_letter_queue": "my-app-events-dlq"
      }
    ]
  }
}

We’ll come back to those consumer options.

Send a message

In your Worker, send a message with the producer binding:

export default {
  async fetch(request, env) {
    // ... create the user ...

    await env.EVENTS.send({ type: 'welcome-email', userId: 123 })

    return Response.json({ ok: true })
  },
}

The send returns as soon as the message is accepted. The user gets their response right away.

Process messages

The consumer is a queue handler in the same Worker. Cloudflare calls it with a batch of messages:

export default {
  async fetch(request, env) {
    // ... producer code from above ...
  },

  async queue(batch, env, ctx) {
    for (const message of batch.messages) {
      const event = message.body

      if (event.type === 'welcome-email') {
        await sendWelcomeEmail(event.userId)
      }

      message.ack()
    }
  },
}

message.body is the object you sent. message.ack() tells Cloudflare you’re done with it, so it won’t be delivered again.

Batching

Notice you get a batch, not one message at a time. That’s on purpose. Processing 10 messages in one go is more efficient than 10 separate runs.

You tune this in the config:

• max_batch_size: how many messages per batch (up to a limit).
• max_batch_timeout: how long to wait to fill a batch before delivering it anyway.

So “10 messages, or whatever’s there after 5 seconds, whichever comes first.”

Retries and the dead letter queue

What if processing fails? Maybe the email service is down for a second.

If your handler throws, or you call message.retry(), Cloudflare delivers the message again later. It keeps trying up to max_retries times.

After that, instead of throwing the message away, it sends it to the dead letter queue (the dead_letter_queue in the config). That’s a separate queue holding the messages that never succeeded, so you can inspect them and figure out what went wrong.

You can retry a single message and keep the rest:

async queue(batch, env, ctx) {
  for (const message of batch.messages) {
    try {
      await process(message.body)
      message.ack()
    } catch (err) {
      message.retry()
    }
  }
}

Why I reach for queues

Anything slow or flaky belongs in a queue: sending email, calling a third-party API, generating a report, syncing data.

The user’s request stays fast, and the background work gets reliable retries for free. The full reference is in the Queues docs.

~~~

Related posts about cloudflare: