惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

U
Unit 42
Microsoft Azure Blog
Microsoft Azure Blog
Engineering at Meta
Engineering at Meta
博客园 - 【当耐特】
人人都是产品经理
人人都是产品经理
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
WordPress大学
WordPress大学
有赞技术团队
有赞技术团队
Blog — PlanetScale
Blog — PlanetScale
酷 壳 – CoolShell
酷 壳 – CoolShell
aimingoo的专栏
aimingoo的专栏
Jina AI
Jina AI
小众软件
小众软件
博客园 - 叶小钗
MongoDB | Blog
MongoDB | Blog
大猫的无限游戏
大猫的无限游戏
博客园 - 聂微东
Y
Y Combinator Blog
云风的 BLOG
云风的 BLOG
I
InfoQ
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Martin Fowler
Martin Fowler
P
Proofpoint News Feed
MyScale Blog
MyScale Blog

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
Your pytest configuration is lying to you. Not loudly but...
Emjey · 2026-05-01 · via DEV Community

Picture this. A test starts failing in CI. You pull the branch locally, run the suite, and it passes. You dig in for twenty minutes before realising the CI run has --capture-screenshots=failed and yours doesn't, so the screenshot that would have told you exactly what broke exists in CI, but you never thought to check because your local run gave you nothing.

Or the reverse: you add --plus-email to your CI command to get the report delivered after each run, forget to strip it out of the local config, and now every developer on the team is getting email on every local test run.

These aren't dramatic failures. They're the quiet kind, where local and CI are technically both running pytest, but they're not running the same thing, and nobody noticed until something went wrong.

This is what test configuration looks like for most teams
The natural starting point is addopts in pyproject.toml:

[tool.pytest]
addopts = "--generate-xml --xml-report=ci.xml"

Enter fullscreen mode Exit fullscreen mode

That works until you realise it applies to everything — local runs, CI, the nightly suite, all of it. There's no way to say "this option only when running in CI" or "this one only locally." It's a single global value. So you either put the CI-specific flags there and accept that local runs generate XML nobody wants, or you leave them out and accept that the command you paste into GitHub Actions is already diverging from your config.

The next move is Makefile targets:

test-ci:
    pytest --generate-xml --xml-report=ci.xml --capture-screenshots=failed tests/

test-local:
    pytest --html-output=local-report --should-open-report=always tests/

Enter fullscreen mode Exit fullscreen mode

That works locally. But make isn't guaranteed to be installed on CI runners, and in practice most teams just paste the raw pytest command into the workflow YAML anyway — calling make test-ci from a GitHub Actions step feels like extra indirection for no real benefit. So you end up maintaining the Makefile target and a duplicate command in your CI config, and they drift apart within a month.

What you actually want is a way to say: here is what a CI run looks like, here is what a local run looks like, here is what nightly looks like named, in one place, in something that travels with the repo and works identically everywhere.

So I added profiles to pytest-html-plus:

[tool.pytest-html-plus.profiles.ci]
html-output = "ci-report"
json-report = "ci.json"
capture-screenshots = "failed"
generate-xml = true
xml-report = "ci.xml"

[tool.pytest-html-plus.profiles.local]
html-output = "local-report"
capture-screenshots = "all"
should-open-report = "always"

[tool.pytest-html-plus.profiles.nightly]
generate-xml = true
xml-report = "nightly.xml"
capture-screenshots = "all"
plus-email = true

Enter fullscreen mode Exit fullscreen mode

Then:

pytest --plus-profile=ci tests/
pytest --plus-profile=local tests/
pytest --plus-profile=nightly tests/

Enter fullscreen mode Exit fullscreen mode

One flag. Each environment gets its own named config. All of it lives in pyproject.toml, committed to the repo, visible to everyone on the team. No Makefile, no duplicated YAML, no addopts compromise.
When you add a new option to the ci profile, every place that runs --plus-profile=ci picks it up — the GitHub Actions workflow, the staging runner, wherever. You change it once.

Why the "just use environment variables" answer doesn't quite work:

PYTEST_ADDOPTS is the other common suggestion. Set it in CI, set a different value locally, done. The problem is it's invisible — it's not in the repo, it's not in a config file a new team member would find, and it requires everyone to have the right shell variable set up on their machine before local runs behave the way CI expects. Environment variables are fine for secrets. They're not a great place to store the definition of what a test run is supposed to look like.

Profiles are checked into source control. They're discoverable. And the error messages are designed to be useful — if you reference a profile that doesn't exist, you get the list of available ones immediately rather than silently falling back to defaults.

One useful detail about how it works

When you pass --plus-profile=ci, the profile args are prepended to your argument list before pytest processes them. Because pytest uses the last occurrence of a repeated flag, anything you add on the command line overrides the profile. So you can use a profile as a base and adjust for a specific run without changing the profile itself:

pytest --plus-profile=ci --json-report=debug-run.json tests/

Enter fullscreen mode Exit fullscreen mode

The profile defines json-report = "ci.json", but your flag wins. Useful for one-off debugging without polluting the shared config.

What this doesn't do

Profiles only cover pytest-html-plus options, things like where the HTML report goes, whether to generate XML, when to capture screenshots. It's not a general-purpose pytest config system. You can't use it to set -n auto or --reruns 2 or anything that isn't a pytest-html-plus flag.
For that, addopts and pyproject.toml's native pytest config are still the right tools. Profiles sit alongside that, not instead of it.

One thing TOML can't do is expand environment variables. So if you want git-branch or git-commit to reflect the actual branch and SHA at runtime — which you almost certainly do in CI, you can't write git-branch = "$CI_BRANCH" and have it work. Instead, pass those as CLI overrides:

pytest --plus-profile=ci --git-branch=$CI_BRANCH --git-commit=$CI_COMMIT_SHA tests/

Enter fullscreen mode Exit fullscreen mode

The profile handles the static options. The dynamic ones come in on the command line. That's consistent with how the override mechanism already works, but it's worth knowing upfront rather than discovering it mid-setup.

The full list of supported keys and type rules are in the usage docs.

pip install pytest-html-plus
pytest --plus-profile=ci tests/

Enter fullscreen mode Exit fullscreen mode

The project is on GitHub at reporterplus/pytest-html-plus. If something doesn't behave the way you expect — especially if your setup has unusual CI runner configurations or custom pyproject.toml structures — open an issue. That's the feedback that actually improves it.