惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

阮一峰的网络日志
阮一峰的网络日志
博客园 - 司徒正美
D
DataBreaches.Net
宝玉的分享
宝玉的分享
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
博客园 - 【当耐特】
人人都是产品经理
人人都是产品经理
博客园 - Franky
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
IT之家
IT之家
博客园 - 三生石上(FineUI控件)
J
Java Code Geeks
腾讯CDC
博客园_首页
The Cloudflare Blog
S
SegmentFault 最新的问题
C
Check Point Blog
美团技术团队
爱范儿
爱范儿
大猫的无限游戏
大猫的无限游戏
Hugging Face - Blog
Hugging Face - Blog
T
The Blog of Author Tim Ferriss
A
About on SuperTechFans
Blog — PlanetScale
Blog — PlanetScale

Datadog | The Monitor blog

Introducing our open source AI-native SAST Instrument and monitor Boomi integration flows with OpenTelemetry and Datadog Not all index scans are equal: How we cut query latency by over 99% Platform engineering metrics: What to measure and what to ignore Integrate Recorded Future threat intelligence with Datadog Cloud SIEM CI/CD security: threat modeling using a MITRE-style threat matrix CI/CD security: How to secure your GitHub ecosystem Ingress NGINX is EOL: A practical guide for migrating to Kubernetes Gateway API Operating agentic AI with Amazon Bedrock AgentCore and Datadog LLM Observability: Lessons from NTT DATA Introducing the Datadog Code Security MCP Capture and analyze custom heatmaps in Session Replay Understand session replays faster with AI summaries and smart chapters Monitor ClickHouse query performance with Datadog Database Monitoring How we designed empathetic alert sounds for on-call engineers Search and act across Datadog to resolve issues faster with Bits Assistant Measure the business impact of every product change with Datadog Experiments Analyzing round trip query latency Configuring JavaScript caches for better performance Introducing Bits AI Dev Agent for Code Security Datadog achieves ISO 42001 certification for responsible AI Monitor Nutanix clusters, hosts, and VMs with Datadog Monitor Juniper Mist in Datadog A new Host Map for modern infrastructure Annotate traces to improve LLM quality with Datadog LLM Observability What’s new in Cloud SIEM: AI-powered investigations, enhanced threat intelligence, and scalable security operations Explore Kubernetes with native OpenTelemetry data Monitor Oracle Fusion Cloud Applications with Datadog Announcing the Datadog Terraform provider v4.0.0 Scaling Kubernetes workloads on custom metrics How to design cloud environments for AI-powered threat analysis
Enrich your on-call experience with observability data at...
Brianne Bujnowski, Daljeet Sandu · 2024-06-26 · via Datadog | The Monitor blog

The stress, sudden disruptions, and high stakes of resolving issues while on call is one of the most challenging aspects of an engineer’s job. Many organizations, from startups to large enterprises, still struggle with their on-call experience, which leads to longer resolution times and lower employee retention rates. Constant context switching, managing multiple tools, and racing against time to resolve issues can cause frustration, burnout, and inefficiency.

Having a single tool to observe your tech stack, detect issues quickly, and page the right people at the right time is crucial. That’s why we’ve introduced Datadog On-Call, now generally available. Datadog On-Call enriches your on-call experience with observability context, mobilizing responders with data-driven pages, service and team organization details, dynamic scheduling and notifications, and deep analytics for fast, purposeful coordination. And as part of Datadog Incident Response, which combines On-Call and Incident Management, teams can monitor, page, and respond to incidents all on one platform. This not only enhances efficiency and reduces stress but also empowers your team to respond to incidents faster and more effectively, ultimately maintaining the reliability and performance of your systems.

In this post, we’ll walk through how Datadog On-Call enables your teams to:

  • Consolidate monitoring and paging into a single platform

  • Break down knowledge silos with clear team and service ownership

  • Ensure timely responses with intuitive scheduling and escalation policies

  • Gain actionable insights from pages with detailed analytics

Consolidate monitoring and paging into a single platform

One of the major frustrations during an on-call shift is the need to juggle multiple tools and platforms to gather all the necessary information. Engineers often have to switch between Datadog and paging systems, which not only wastes precious time but also increases the risk of missing critical details.

With Datadog On-Call, the seamless integration of monitoring and paging ensures that you receive real-time notifications directly from the same platform where you can analyze the issue and collaborate throughout remediation. This workflow eliminates the inefficiencies caused by context switching, enabling your team to detect and respond to incidents faster. By having all the tools you need in one place, Datadog On-Call enhances productivity, reduces the stress of managing multiple systems, and improves overall incident response times.

Easily manage and take action on pages from one view.

For example, let’s say you’re an on-call backend engineer using Datadog and receive a page at three in the morning. With Datadog On-Call, when the alert is triggered, you are paged via push notification to access the Datadog Mobile App. From there, you can review the impact of the alert alongside relevant observability data and effectively triage the alert from your phone. If the impact is severe enough, with Incident Response you can go further by declaring an incident from the mobile app and triggering workflow automations that quickly implement potential resolutions. This entire process, from page to incident resolution, can be done on-the-go from one platform.

Break down silos with clear service and team ownership

Organizations often end up with redundant service configurations when they have separate paging and monitoring tools. This fragmentation can lead to confusion about service ownership and responsibility, making it difficult to determine who should be paged for specific issues. The lack of clear team ownership results in delays and prolonged incidents as engineers scramble to identify the right contact.

Datadog On-Call addresses these challenges with a team-centric design that shows clear service and team ownership. With Datadog On-Call, you can associate a team with any service to reduce redundant configurations and ensure services are mapped to the appropriate owners for a single, at-a-glance source of truth. Additionally, after a page, on-call engineers can immediately see the upstream and downstream impact of issues with the Datadog Service Catalog and bring further details to the attention of the right owners.

Ensure timely responses with intuitive scheduling and escalation policies

Effective scheduling and escalation policies are essential for managing on-call duties without overburdening your team. Traditional scheduling methods can be cumbersome, leading to an uneven distribution of on-call shifts and an increased risk of burnout.

Schedule list of all on-call engineers on your team.

Datadog On-Call simplifies this process with intuitive scheduling tools that make it easy to create and manage on-call rotations. The On-Call page supports quality-of-life improvements such as drag-and-drop or live schedule previews, allowing you to set up schedules that ensure fair distribution of duties, prevent fatigue, and maintain a balanced workload.

Creating and keeping an on-call schedule is seamless.

Beyond its scheduling benefits, Datadog On-Call’s robust escalation policies ensure that pages are promptly addressed. If the primary on-call engineer is unavailable or does not acknowledge a page, the next available team member is automatically notified. By implementing these intuitive scheduling and escalation features, Datadog On-Call helps maintain high levels of responsiveness and reliability in your incident management process.

Gain actionable insights from pages with detailed analytics

Reviewing past pages is critical for teams to understand the root causes of future incidents and identify opportunities for improvement. These reviews help teams answer core questions such as: What triggered the alert? How effective was the response? Were there any delays in detection or acknowledgment? What can be done to prevent similar incidents in the future? By conducting thorough page reviews, teams can analyze their incident response processes and make data-driven decisions to enhance their workflows.

Detailed analytics of all your pages.

Datadog On-Call provides detailed analytics that make page reviews more insightful and productive. Teams can easily access metrics such as the number of pages received, the time taken to respond to alerts, and the duration of the incident. These metrics enable teams to pinpoint inefficiencies and areas for improvement. For example, if recurring issues are identified, teams can adjust monitoring thresholds or update runbooks to ensure quicker resolutions in the future.

Improve your on-call experience today

Datadog On-Call brings monitoring, paging, and incident resolution into one unified platform, helping on-call engineers see which team members are most active in resolving issues and which services cause the most operational load. This reduces on-call burden and continuously improves team processes, making work more efficient and effective.

Try out Datadog On-Call for your team today, or use it as part of Datadog Incident Response for comprehensive monitoring, paging, and incident resolution. If you’re not already using Datadog, get started today with a 14-day free trial.