Per-PR ephemeral email inboxes for E2E tests in GitHub Ac...

Your password-reset flow needs an inbox to test against. Your invitation flow too. Your email-verification gate too. The classic setup is a "test+pr-1234@yourdomain.com" alias on a shared mailbox, polling Gmail's API, hoping nothing else lands while the test runs. It is fragile, it leaks state across PRs, and your credentials live in CI.

A managed agent account flips this. Each PR gets a fresh inbox, lives only for the duration of the test run, and tears itself down at the end.

What this gives you

One inbox per PR, isolated from every other test
Real send/receive — not a mock
No shared Gmail credentials in CI
Cleanup is one command per inbox
Parallel test workers do not collide

The full GitHub Action

# .github/workflows/e2e.yml
name: E2E with ephemeral inbox

on:
  pull_request:
    branches: [main]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Nylas CLI
        run: curl -fsSL https://cli.nylas.com/install.sh | bash
        env:
          NYLAS_INSTALL_DIR: ${{ github.workspace }}/.nylas

      - name: Authenticate
        run: $NYLAS_INSTALL_DIR/bin/nylas auth config --api-key ${{ secrets.NYLAS_API_KEY }}

      - name: Create ephemeral inbox
        id: inbox
        run: |
          EMAIL="e2e-pr${{ github.event.number }}-run${{ github.run_id }}@yourapp.nylas.email"
          $NYLAS_INSTALL_DIR/bin/nylas agent account create "$EMAIL" --json > inbox.json
          echo "email=$EMAIL" >> $GITHUB_OUTPUT
          echo "grant_id=$(jq -r .id inbox.json)" >> $GITHUB_OUTPUT

      - name: Run Playwright
        run: pnpm test:e2e
        env:
          E2E_INBOX: ${{ steps.inbox.outputs.email }}
          E2E_GRANT_ID: ${{ steps.inbox.outputs.grant_id }}

      - name: Tear down inbox
        if: always()
        run: $NYLAS_INSTALL_DIR/bin/nylas agent account delete ${{ steps.inbox.outputs.grant_id }} --yes

The if: always() on teardown makes sure the inbox is removed even when tests fail.

How the test reads mail

// tests/auth.spec.ts
import { test, expect } from '@playwright/test'
import { execFileSync } from 'node:child_process'

test('password reset email lands and link works', async ({ page }) => {
  const inbox = process.env.E2E_INBOX!
  const grantId = process.env.E2E_GRANT_ID!

  // 1. Trigger password reset
  await page.goto('/forgot-password')
  await page.fill('[name=email]', inbox)
  await page.click('button:has-text("Send reset link")')

  // 2. Poll the inbox for up to 30 seconds
  const link = await pollForResetLink(grantId, 30_000)
  expect(link).toMatch(/\/reset\/[a-f0-9-]+/)

  // 3. Click the link, verify the new-password form
  await page.goto(link)
  await expect(page).toHaveURL(/\/reset\//)
})

function pollForResetLink(grantId: string, timeoutMs: number): string {
  const deadline = Date.now() + timeoutMs
  while (Date.now() < deadline) {
    const out = execFileSync('nylas', [
      'email', 'list', '--grant', grantId, '--unread', '--limit', '5', '--json'
    ], { encoding: 'utf-8' })
    const messages = JSON.parse(out)
    for (const m of messages) {
      const match = (m.snippet || '').match(/https:\/\/[^\s]+\/reset\/[a-f0-9-]+/)
      if (match) return match[0]
    }
    require('node:child_process').execSync('sleep 2')
  }
  throw new Error('reset email never arrived')
}

Why this beats the alternatives

Approach	Per-PR isolation	No shared creds	Real inbox	Setup time
Mailosaur	✅	✅	✅	Paid SaaS
MailHog (self-hosted)	❌	✅	❌ (dev only)	Run a daemon
Gmail "+pr-1234" alias	Partial	❌	✅	OAuth setup
Agent accounts	✅	✅	✅	Two CLI commands

Mailosaur is excellent and used by many teams. The trade is the SaaS contract. Agent accounts ride your existing Nylas plan.

Parallel workers

GitHub Actions runs multiple PRs concurrently. Two PRs landing at the same time would clobber a shared inbox. Per-PR provisioning makes that a non-issue:

PR #1234 → e2e-pr1234-run-9999@yourapp.nylas.email
PR #1235 → e2e-pr1235-run-1000@yourapp.nylas.email

If you parallelise inside a single PR (e.g., matrix strategy), include the matrix index in the email:

strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: |
      EMAIL="e2e-pr${{ github.event.number }}-shard${{ matrix.shard }}@yourapp.nylas.email"
      ...

Cost notes

Each agent account is one grant. Most plans bill on grants, so a 50-PR-a-week rate is 50 grants created and destroyed weekly. Cleanup is mandatory or you accumulate dead accounts. The if: always() step above takes care of that.

A real-world catch I hit

GitHub Actions runners do not have nylas on PATH after curl install.sh | bash — the binary is at $NYLAS_INSTALL_DIR/bin. Either add to PATH explicitly or invoke with the full path, as in the workflow above. I forgot this twice.

Next steps

E2E email testing with Playwright — the hands-on guide for password-reset and verification flows
Receive email without an SMTP server — production agent-account patterns
Create an AI agent email identity — full agent account walkthrough
Full command reference

推荐订阅源

DEV Community