惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Martin Fowler
Martin Fowler
D
DataBreaches.Net
F
Fortinet All Blogs
阮一峰的网络日志
阮一峰的网络日志
博客园_首页
Apple Machine Learning Research
Apple Machine Learning Research
H
Help Net Security
M
MIT News - Artificial intelligence
美团技术团队
人人都是产品经理
人人都是产品经理
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
The Cloudflare Blog
有赞技术团队
有赞技术团队
L
LangChain Blog
博客园 - Franky
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
博客园 - 【当耐特】
S
SegmentFault 最新的问题
V
Visual Studio Blog
Blog — PlanetScale
Blog — PlanetScale
Hugging Face - Blog
Hugging Face - Blog
B
Blog
I
InfoQ

Datadog | The Monitor blog

Introducing our open source AI-native SAST Instrument and monitor Boomi integration flows with OpenTelemetry and Datadog Not all index scans are equal: How we cut query latency by over 99% Platform engineering metrics: What to measure and what to ignore Integrate Recorded Future threat intelligence with Datadog Cloud SIEM CI/CD security: threat modeling using a MITRE-style threat matrix CI/CD security: How to secure your GitHub ecosystem Ingress NGINX is EOL: A practical guide for migrating to Kubernetes Gateway API Operating agentic AI with Amazon Bedrock AgentCore and Datadog LLM Observability: Lessons from NTT DATA Introducing the Datadog Code Security MCP Capture and analyze custom heatmaps in Session Replay Understand session replays faster with AI summaries and smart chapters Monitor ClickHouse query performance with Datadog Database Monitoring How we designed empathetic alert sounds for on-call engineers Search and act across Datadog to resolve issues faster with Bits Assistant Measure the business impact of every product change with Datadog Experiments Analyzing round trip query latency Configuring JavaScript caches for better performance Introducing Bits AI Dev Agent for Code Security Datadog achieves ISO 42001 certification for responsible AI Monitor Nutanix clusters, hosts, and VMs with Datadog Monitor Juniper Mist in Datadog A new Host Map for modern infrastructure Annotate traces to improve LLM quality with Datadog LLM Observability What’s new in Cloud SIEM: AI-powered investigations, enhanced threat intelligence, and scalable security operations Explore Kubernetes with native OpenTelemetry data Monitor Oracle Fusion Cloud Applications with Datadog Announcing the Datadog Terraform provider v4.0.0 Scaling Kubernetes workloads on custom metrics How to design cloud environments for AI-powered threat analysis
A FinOps engineer’s guide to governing custom metrics
2025-12-19 · via Datadog | The Monitor blog

This guest blog post is authored by Dieter Matzion, a seasoned cloud practitioner who has operated exclusively in public cloud environments since 2013, with experience at leading technology companies including Google, Netflix, Intuit, and Roku.

Custom metrics play a crucial role in enabling teams to monitor their applications and businesses. The flexibility of these metrics allows engineers to measure what matters most to their domain. However, as organizations scale, it becomes increasingly important to apply proactive governance. Establishing cost awareness and standardization across teams helps ensure that custom metrics deliver real business value within expected budgets.

In this guide, we’ll walk through how I approach managing custom metrics in Datadog by using data-driven insights, automation, and collaboration between engineering and finance teams.

Building a cost-conscious culture

The foundation of any cost optimization initiative is the alignment between finance and engineering. Although centralized FinOps teams can drive strategic direction, everyone in the organization needs to see themselves as a stakeholder in managing cloud costs.

To achieve this alignment, it’s essential to establish mutually agreed-upon cost targets tied to measurable business outcomes. One effective way to do this is through unit economics, linking custom metrics usage to revenue-driving or cost-saving activities. This technique not only aligns incentives across teams but also creates a shared language between technical and financial stakeholders.

Understanding Datadog spend

For Datadog, the cost of custom metrics is driven by cardinality, i.e., the number of unique timeseries generated by tag value combinations across all metrics. One effective governance practice is adopting standardized metric-naming conventions. Embedding hierarchical context into metric names helps ensure clarity and traceability across teams. Consider the following example:

ACME.data_activation.lambda.lat_setter.aerospike_read.mean

ACME.data_activation.lambda.lat_setter.aerospike_read.median

ACME.data_activation.lambda.lat_setter.aerospike_read.p99

ACME.data_activation.lambda.lat_setter.aerospike_read.stddev

Each segment of the metric name provides specific contextual information:

  • The ACME segment identifies the application.
  • The data_activation segment refers to a module within the application.
  • The lambda segment designates the serverless function responsible for executing the module.
  • The lat_setter segment represents a latency histogram bucket setter used internally to track and classify operation latencies.
  • The aerospike_read segment specifies the type of database operation being monitored—in this case, a read from Aerospike, a high-performance, distributed NoSQL database.
  • The mean, median, p99, and stddev segments indicate the specific statistical measurements being collected for the latency metric.

When paired with relevant tags, this type of naming convention provides additional context that helps engineers efficiently organize, filter, and analyze metrics. But since tags influence cardinality, and therefore spend, it’s important to review tags and ensure they’re adding value.

Datadog provides a number of tools to help with optimization initiatives—from visibility and attribution to value-based governance. Using Datadog’s out-of-the-box usage metrics, I created a dashboard that tracks overall spend, monthly usage trends, and usage broken down by team. I review this dashboard weekly so that I can catch any abnormalities early on and notify responsible teams to take action. I also hold monthly finance reviews with only the teams who are grossly under or over budget.

Custom metrics usage, with sample data
Custom metrics usage dashboard showing monthly spend, account trends, and team breakdown
Custom metrics usage, with sample data

Identifying metrics for optimization

Besides notifying teams to optimize their metrics, I was able to safely optimize them myself since Datadog provides visibility into a metric’s relative utility. During my investigation into opportunities for optimizing custom metrics, I discovered that the many had not been queried within the past 90 days. Engineering teams across the organization had been proactively provisioning custom metrics for a wide range of potential scenarios—often with the intention of supporting future debugging or incident analysis. But in practice, these custom metrics weren’t being used.

Custom metrics by query activity, with sample data
Metrics summary table listing metric names, tag status, query activity, and last configured dates
Custom metrics by query activity, with sample data

Analysis revealed that only a subset of these metrics were actually being used in practice. After confirming low-usage patterns, I worked with engineering teams to either deprecate those metrics or remove unnecessary tags, reducing unnecessary volumes while preserving the observability needed for effective operations and troubleshooting.

Optimizing custom metrics by using Metrics without Limits™

Using Metrics without Limits enables you to choose which tags to include and exclude for indexing. In particular, when you select the option to include tags, Datadog automatically suggests tags that have been queried recently, helping you focus on what’s actively used. This feature enabled me to exclude unused tags from indexing, significantly reducing the volume of indexed custom metrics.

The Include Tags configuration option
Manage tags dialog showing tag allowlist and estimated reduction in indexed custom metrics
The Include Tags configuration option

Another option I used was Configure and Remove Tags. When this option is selected, Datadog automatically removes unused tag combinations from indexing across all metrics. This significantly reduces manual effort and makes large-scale tag optimization more efficient and scalable. In my case, the option lowered the number of indexed metrics from over 900,000 to just above 100,000.

The Configure and Remove Tags option
Confirmation dialog to remove tags from unused metrics queried within the last 90 days
The Configure and Remove Tags option

Govern your custom metrics today

By aligning engineering and finance, adopting naming and tagging standards, and using Datadog’s optimization features, FinOps teams can create more efficient, sustainable monitoring practices for custom metrics. Whether you work in finance or engineering, Datadog makes it easier to achieve these goals and manage your observability spend.

To learn more, check out Datadog’s Best Practices for Custom Metrics Governance guide. And if you’re new to Datadog, sign up for a 14-day free trial.