惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Apple Machine Learning Research
Apple Machine Learning Research
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
G
Google Developers Blog
博客园 - 司徒正美
J
Java Code Geeks
aimingoo的专栏
aimingoo的专栏
A
About on SuperTechFans
博客园 - 三生石上(FineUI控件)
WordPress大学
WordPress大学
T
The Blog of Author Tim Ferriss
D
Docker
大猫的无限游戏
大猫的无限游戏
D
DataBreaches.Net
腾讯CDC
V
Visual Studio Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
C
Check Point Blog
M
MIT News - Artificial intelligence
Jina AI
Jina AI
I
InfoQ
雷峰网
雷峰网
The Cloudflare Blog
美团技术团队
Engineering at Meta
Engineering at Meta

Datadog | The Monitor blog

Introducing our open source AI-native SAST Instrument and monitor Boomi integration flows with OpenTelemetry and Datadog Not all index scans are equal: How we cut query latency by over 99% Platform engineering metrics: What to measure and what to ignore Integrate Recorded Future threat intelligence with Datadog Cloud SIEM CI/CD security: threat modeling using a MITRE-style threat matrix CI/CD security: How to secure your GitHub ecosystem Ingress NGINX is EOL: A practical guide for migrating to Kubernetes Gateway API Operating agentic AI with Amazon Bedrock AgentCore and Datadog LLM Observability: Lessons from NTT DATA Introducing the Datadog Code Security MCP Capture and analyze custom heatmaps in Session Replay Understand session replays faster with AI summaries and smart chapters Monitor ClickHouse query performance with Datadog Database Monitoring How we designed empathetic alert sounds for on-call engineers Search and act across Datadog to resolve issues faster with Bits Assistant Measure the business impact of every product change with Datadog Experiments Analyzing round trip query latency Configuring JavaScript caches for better performance Introducing Bits AI Dev Agent for Code Security Datadog achieves ISO 42001 certification for responsible AI Monitor Nutanix clusters, hosts, and VMs with Datadog Monitor Juniper Mist in Datadog A new Host Map for modern infrastructure Annotate traces to improve LLM quality with Datadog LLM Observability What’s new in Cloud SIEM: AI-powered investigations, enhanced threat intelligence, and scalable security operations Explore Kubernetes with native OpenTelemetry data Monitor Oracle Fusion Cloud Applications with Datadog Announcing the Datadog Terraform provider v4.0.0 Scaling Kubernetes workloads on custom metrics How to design cloud environments for AI-powered threat analysis
Ensure high service availability with Datadog Service Man...
Addie Beach, Brianne Bujnowski, Cansu Berkem · 2024-11-01 · via Datadog | The Monitor blog

Adopting a cloud-based, distributed architecture may help your organization scale quickly, but it can also add complexity. Correlating telemetry, security signals, and alerts across services often proves difficult, resulting in slower issue remediation. Additionally, when something goes wrong, figuring out who to contact—for example, the on-call responder or the service owner— may become needlessly time-consuming. To navigate these challenges, you need a unified service management approach that brings together monitoring, DevOps, IT operations, and app development tools for effective communication and fast issue resolution.

Datadog’s Service Management offerings help you quickly detect issues within your system and take action on them without leaving the platform. By drawing on service data from throughout the Datadog platform, features such as Incident, Case, and Event Management enable you to seamlessly pivot from viewing observability data that details issues within your services to contacting service owners and responders for troubleshooting. As a result, you can improve reliability while preventing tool sprawl and streamlining engineering processes.

In this post, we’ll explore how Datadog Service Management can help you:

  • Quickly detect and remediate issues from a centralized hub

  • Triage and track issues, tasks, and to-do items

  • Easily correlate service events and reduce alert fatigue

Incident response often involves repeatedly pivoting between a variety of tools, taking precious minutes away from time-sensitive incident remediation. Responding to an issue may involve being paged in one platform, opening the ticket in another, pivoting to a monitoring platform to view troubleshooting data, and then accessing yet another tool to track down relevant service owners. This piecemeal process makes it challenging to detect and triage issues with multi-service impact, especially if they require you to coordinate response activities across teams.

By contrast, Datadog Service Management gives you a centralized hub for responding to problems within your services, with alerting, paging, response planning, and automation functionality all in the same platform. When an issue is detected, Datadog On-Call quickly notifies relevant teams of the problem and provides essential context by drawing on observability data. Responders receive detailed alerts with real-time service data, and can easily create incidents directly from these alerts. Datadog Incident Management then immediately helps you kick off the response process with built-in automated actions, including customizable updates to stakeholders sent via the communication platform of your choice. Incident Management also helps you keep track of response actions by capturing your to-do items, Zoom calls, Slack conversations, and graphs.

The Datadog On-Call Summary page, with a list of the latest pages displayed.

Additionally, you can use Bits AI with Incident Management to simplify your response process. You can stay up-to-date on ongoing issues without leaving your collaboration tools—including Slack and Teams—by using auto-generated summaries, natural language prompts, and ticket management features within incident channels. Once the incident is resolved, Datadog Incident Management can generate comprehensive postmortems that can help you remediate—and even prevent—similar issues in the future.

Triage and track issues, tasks, and to-do items

After an incident has been closed, there are still follow-up items to complete, other issues to investigate, and day-to-day maintenance tasks to work on. Proactively handling these activities helps you prevent future incidents as well as track improvements on specific services.

With Case Management, Datadog gives you a centralized location for organizing your tasks, to-do items, investigations, and follow-ups. Within each case, you can track which team members have been assigned to an issue as well as any remediation actions they’ve taken. You can easily automate case creation from tickets in other platforms via our bidirectional Jira and ServiceNow integrations, helping you sync your data. Additionally, Case Management integrates with the rest of the Datadog platform, enabling teams to enrich cases with graphs, logs, alerts, and notebooks.

A case within Datadog Case Management, showing details such related events and the assigned responder.

Datadog not only integrates built-in automation into your incidents and cases to help you troubleshoot faster but also provides customizable tools that enable you to automate critical actions elsewhere in your system. Datadog Workflow Automation provides easy-to-create templates you can use to implement new workflows without coding, as well as granular logical, conditional, and branching options for more complex processes.

Workflow Automation can handle both simple operational activities and more complex response processes. One example of the latter is when your system suffers from larger, recurring issues with simple temporary patches, such as complicated memory leaks that can be kept at bay with Kubernetes pod restarts. While you work to pinpoint and remediate the root cause—which could take weeks—you can create a workflow that automatically restarts the affected pods when they run out of memory and crash. In addition to giving you more time to focus on fixing the problem, this can reduce the number of late-night pages for quick tasks, helping cut down on burnout. Alternatively, for less straightforward issues, you can even configure your workflows to message the relevant responders and create incidents or cases, with Datadog or elsewhere.

Easily correlate service events and reduce alert fatigue

Keeping modern cloud systems highly available and performant requires teams to track a wealth of service activity, including system errors, routine updates, and suspicious user behavior. With so many signals coming from different parts of your stack, it can be challenging to identify legitimate incidents among the noise. This can lead to both alert fatigue among your responders and delayed incident response efforts.

To help you address these challenges, Datadog Event Management enables you to centralize and consolidate alerts and events from your entire stack—including ones from third-party sources. Strengthened by machine learning, Datadog automatically correlates related alerts based on tag commonality, service topology, alert entropy, and other heuristics so you can understand an issue’s full breadth of impact and expedite time to assign. You can add critical, business-specific data—such as ownership or location details—to these alerts using Enrichment processors and include changes that occurred around the time of your correlated alerts to simplify root cause analysis. When you do decide to take action on an alert, Event Management helps you automate triage and remediation by sending notifications to responders and stakeholders, executing remediation workflows, and escalating cases within Case Management to Incident Management.

The Overview page in Datadog Event Management, with a graph showing a decrease in alerts due to correlation displayed.

Centralize your service management strategy with Datadog

With a centralized platform that helps you manage everything from on-call responders to case creation, Datadog helps you close the circle with your service management strategy. You can easily triage alerts, manage tickets, and access events for your services, with critical observability data available every step of the way.

Check out our documentation to learn more about Incident Management, Case Management, Event Management, and other Datadog Service Management features. Or, if you’re new to Datadog, you can sign up for a 14-day free trial.