Guys, I need your help 😅
As I’ve noticed, many QA Engineers (also devs) do not measure how stable or fast their tests are. Even teams that do care about test observability often build their own internal tooling to analyze test statistics.
People sometimes ask me how they can organize their own analytical system for test observability. But I never had a clear answer. Just "develop your own system". This approach might work for large teams, but it can be expensive for small and medium teams. Googling didn't help me. I could not find any tools that I would confidently recommend to others. And...
I decided to create a simple solution for people who use Python and pytest. It is a pytest-plugin designed to improve test observability without requiring a TMS or other paid solutions. It exports test results into ClickHouse.
Key features:
- Automatic CI detection
- Allure report support
- Basic Grafana configuration provided out of the box
- No impact on pytest execution
- No additional overhead if export to ClickHouse is disabled
- Integrates with pytest
I’m looking for people who can test it, provide feedback, and suggest feature requests. Let’s build a useful tool for the QA community together 🙃
How to test it locally
If you want to see the dashboards, I added a docker-compose file so you don't need to configure a DB from scratch.
git clone https://github.com/shakhov-dmitrii/pytest-test-observer.git
cd pytest-test-observer
docker compose up -d
Then just install the plugin and run your tests:
pip install "pytest-test-observer[allure]"
pytest --ch-url=localhost:8123 --ch-table=pytest_results
You can check the results in Grafana at http://localhost:3000/d/pytest-test-observer-overview (admin / admin).
Here is the repository: https://github.com/shakhov-dmitrii/pytest-test-observer
Having historical metrics for tests has been a massive quality of life upgrade, but I want to make it better.