惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

大猫的无限游戏
大猫的无限游戏
H
Hacker News: Front Page
T
The Blog of Author Tim Ferriss
WordPress大学
WordPress大学
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Blog — PlanetScale
Blog — PlanetScale
Stack Overflow Blog
Stack Overflow Blog
F
Fortinet All Blogs
H
Help Net Security
罗磊的独立博客
D
DataBreaches.Net
MyScale Blog
MyScale Blog
美团技术团队
人人都是产品经理
人人都是产品经理
L
LangChain Blog
M
MIT News - Artificial intelligence
C
Check Point Blog
GbyAI
GbyAI
B
Blog RSS Feed
Microsoft Azure Blog
Microsoft Azure Blog
Y
Y Combinator Blog
雷峰网
雷峰网
Last Week in AI
Last Week in AI
F
Full Disclosure
量子位
V
Visual Studio Blog
Google DeepMind News
Google DeepMind News
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
S
SegmentFault 最新的问题
云风的 BLOG
云风的 BLOG
H
Hackread – Cybersecurity News, Data Breaches, AI and More
P
Proofpoint News Feed
爱范儿
爱范儿
A
About on SuperTechFans
MongoDB | Blog
MongoDB | Blog
腾讯CDC
博客园 - 【当耐特】
U
Unit 42
Martin Fowler
Martin Fowler
NISL@THU
NISL@THU
B
Blog
T
The Exploit Database - CXSecurity.com
Apple Machine Learning Research
Apple Machine Learning Research
L
Lohrmann on Cybersecurity
P
Proofpoint News Feed
有赞技术团队
有赞技术团队
C
CERT Recently Published Vulnerability Notes
The GitHub Blog
The GitHub Blog
T
Threatpost

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant Common SOC 2 Failures (Real World) Stop Vibe-Checking Your AI App: A Practical Guide to Evals How to Use SonarQube and SonarScanner Locally to Level Up Your Code Quality Your Next To-Do App Is Dead — I Replaced Mine with an OpenClaw AI Sign a Nostr event in 60 lines of Python using coincurve — no nostr-sdk, no nbxplorer, no rust toolchain ITGC Audit Explained Like You’re in Big 4 Patch Tuesday abril 2026: Microsoft parcha 163 vulnerabilidades y un zero-day en SharePoint Stop scraping everything: a better way to track competitor price changes Listing on MCPize + the Official MCP Registry while routing payments OUTSIDE the marketplace — how I kept 100% of my x402 revenue Building an AI-Powered Risk Intelligence System Using Serverless Architecture Why We Ripped Function Overloading Out of Our AI Toolchain Testing AI-Generated Code: How to Actually Know If It Works SaaS Churn Is Killing Your Business. Here Is What to Do About It (Without a Support Team) The Speed of AI Is No Longer Linear - And Self-Improving Models Are Why How to Implement RBAC for MCP Tools: A Practical Guide for Engineering Teams From Standard Quote to Persuasive Proposal: AI Automation for Arborists I built a CLI that scaffolds complete multi-tenant SaaS apps Axios CVE-2025–62718: The Silent SSRF Bug That Could Be Hiding in Your Node.js App Right Now The dashboard that ended our friendship Data Pipelines Explained Simply (and How to Build Them with Python) The Hidden Cost of AI Systems Nobody Talks About. undefined vs undeclared, and how typeof behaves Switching from file-based jobs to NATS/Kafka in Rust without changing code io_uring Adventures: Rust Servers That Love Syscalls Why Agentic AI is Killing the Traditional Database The POUR principles of web accessibility for developers and designers Quantum Neural Network 3D — A Deep Dive into Interactive WebGL Visualization How To Install Caveman In Codex On macOS And Windows Automation Pipeline Reliability: Why Your Workflow Breaks When Nobody Is Watching I Built an 'Open World' AI Coding Agent — It Works From ANY Folder From Freelancing to Product: A Tech Service Company's SaaS Transformation China's AI Giants: Adding Tencent Hunyuan & ByteDance Doubao to AI University (74 Providers) On the Vibe Coders and Their Lies clerk: Auto-Summarize Your Claude Code Sessions AI Weekly — 2026/04/10–04/17 | The Model Lockdown Is Here, but the Toolchain Is the Real Battleground AI 週報 — 2026/04/10–2026/04/17 模型封鎖潮來了,但工具鏈才是真戰場 Maybe this is how Open-Source apps are born... 🚀 Fine-Tune LLMs with LoRA and QLoRA: 2026 Guide tRPC v11 + Next.js App Router: End-to-End Type Safety Without the Boilerplate ShadCN UI in 2026: Why I Stopped Installing Component Libraries and Started Owning My Components SaaS Billing in React Server Components: Stripe + Supabase Without a Single `useEffect` Join our DEV Weekend Challenge — $1,000 in Prizes Across TEN winners! Submissions Due April 20 at 6:59 AM UTC. Implementing FSRS Spaced Repetition in Flutter + Supabase — Adding Memory Science to an AI Learning App "I Texted My Localhost From the Train — Claude Code Fixed the Bug Before I Got Home" I Built a Sales Prep AI and It Went Deeper Than Expected Design to Code #2: One JSON, Eleven Outputs Solving the 100M-Row Problem: A Summary Table Pattern for High-Volume Push Notification Logs Flutter Web With Wasm: What Actually Changes For Developers I Built 50 Royalty-Free Soundtracks for My Side Project in a Weekend Using AI Music Generation The Vibe Coding Security Checklist: 7 Things to Check Before You Ship Stop Letting Googlebot Guess Fix Your React App's SEO Right Desconstruindo o Streaming do LinkedIn: Como Criar um Engine de Extração de Vídeo de Alta Performance com HLS e FFmpeg (EDA Part-1) EDA (Exploratory Data Analysis) Explained With Real Life — Why Looking at Your Data Is the Most Important Step in Machine Learning Brand Relationship Management at Scale: Our 4-Touch Outreach System for 200+ Brands Why String.fromEnvironment() Might Return an Empty String in Dart JGuardrails 1.0.0 — Hardening Java LLM Apps Against Jailbreaks, Toxicity, and Prompt Injection Plan and Schedule a Full Week of Threads Content From One Claude Conversation Coding Cat Oran Ep3, Five Tables Changed Everything Updated: BFF Pattern I'm done watching freelancers get buried by 200 proposals. So I'm building the alternative. This is my first post BFS Algorithm in Java Step by Step Tutorial with Examples Tracking LLM Pricing Monthly: An Open Dataset for 22 AI Models How We Measure Content ROI on a Comparison Site: Revenue Attribution Without Perfect Data Introducing Nova AI Ops: The AI-Native Operating System for SRE Teams I built a free desktop video downloader for Windows — Grabbit How Talkie OCR Helps Vision-Impaired & Dyslexic Users Read the World Around Them VRCFaceTracking安装和iPhone面捕配置教程,有bug Even CrowdStrike Can't See Your Agents The Automation Gold Rush: What n8n Workflows and Claude Are Opening Up for Developers Right Now
Why Your Logs Are Useless Without Traces
Arthur · 2026-05-23 · via DEV Community

It is three in the morning, the on-call rotation is awake, and the logs scroll past at a rate the eye cannot track. Ten thousand identical lines reading "ERROR Request failed: Connection timeout" appear in the last fifteen minutes. The timestamps are dense, the request paths blurred, the causal chain absent. Somewhere in the system, a downstream call to an inventory service is failing. The log file does not, in any column, tell anyone which downstream call, which upstream caller, which user request started the cascade, or which retry attempt happens to be the one currently scrolling.

I want to take that scenario seriously, because it is not a logging-quality problem. The logs in question are well-formatted, well-timestamped, and well-aggregated. The team is doing all the things the 2014-vintage advice columns recommended. The problem is structural: a log line is the wrong unit of analysis for the failure they are looking at, and no quantity of better log lines will turn the wrong unit of analysis into the right one. The unit they need is the trace.

What a log line actually is

A structured log entry answers a specific question: "what did this service observe at this moment." It is local to the service, local to the moment, and — by design — has no native concept of where in a wider request lifecycle it sits. In a monolith this is a survivable limitation; the entire request runs in one process, every log line shares an in-memory request context, and a request_id field is enough to grep the picture together after the fact.

In a microservice deployment of any sophistication, the assumption breaks down. A user request that hits a Kubernetes ingress at the edge typically traverses five to twenty internal services before producing a response: an auth gateway, a session-resolution service, two or three domain APIs, a feature-flag layer, several backing data stores, possibly a recommendation service, possibly a billing path. Each of those services emits its own log lines, often to its own log destination, often without a shared correlation field. The request did happen; nobody recorded the shape of it.

The log-correlation problem isn't fixable inside the log abstraction. A log line cannot, by construction, contain information about the call graph it sat inside, because the call graph wasn't visible to the service writing the line. Someone has to record the call graph separately, and that someone is a different signal class entirely.

What a trace is

A trace is the structural answer to "what happened across services for one request." It is a tree of spans, where each span represents a unit of work — a service call, a database query, a cache lookup, an outbound HTTP request — and parent-child relationships preserve the causal nesting. A unique trace_id propagates from the top of the tree (typically the edge ingress) through every hop; each span carries the parent span's ID, plus its own span ID, plus a small bag of attributes (HTTP method, query params, error code, business identifiers).

Rendered visually, a trace is a waterfall: time on the horizontal axis, services and operations on the vertical, each span a coloured bar whose width is its duration. The slow span is the wide one. The failed span is red. The interesting question — "which of the twenty hops in this request consumed the time, and which one returned the error?" — is one screen, not twenty grep commands.

This is not a logging upgrade. It is a different signal class, with a different unit of analysis (the request) than the log signal (the moment), and a different storage model (a tree per request) than the log storage model (a stream per service). The two are complementary, not interchangeable.

The standard you can rely on

Distributed tracing as a discipline is older than most engineers writing about it think. Google's 2010 Dapper paper, by Sigelman and colleagues, is the canonical reference; Twitter open-sourced Zipkin in 2012 as a Dapper-inspired implementation, and Uber open-sourced Jaeger in 2017 on similar lineage. For most of the 2010s, however, the operational reality was vendor-specific: each APM (Datadog, New Relic, AppDynamics, Dynatrace) shipped its own SDK, and instrumenting an application meant choosing a vendor and accepting that the instrumentation work was, structurally, lock-in.

The standardisation arrived in two pieces, both of which are worth pausing on.

The W3C published the Trace Context Recommendation on 6 February 2020, defining a vendor-neutral wire format for propagating trace IDs across HTTP service boundaries. The spec is small and unglamorous — a traceparent header carrying the trace ID, parent span ID, and sampling flags, plus an optional tracestate header for vendor-specific context. Most major HTTP clients and frameworks now respect it as a matter of course.

OpenTelemetry, the merger of OpenTracing and OpenCensus, was accepted to the CNCF Sandbox in May 2019 and moved to Incubating maturity on 26 August 2021, where it remains as of mid-2026. The project ships SDKs for the major languages (Node.js, Python, Java, Go, .NET, Rust, Ruby, PHP), an OTLP wire protocol, and a Collector binary that brokers between application-side instrumentation and any compliant backend. The SDKs include automatic-instrumentation libraries that wire framework-level telemetry without code changes — HTTP servers, ORMs, RPC clients, and message-queue libraries instrument themselves at load time.

The practical consequence is that the 2010s pattern of "pick an APM and live with their SDK" has been replaced by "instrument with OpenTelemetry, ship the OTLP traffic to whichever backend you can afford this quarter." Jaeger, Grafana Tempo, Honeycomb, Datadog APM, New Relic, Splunk Observability, Elastic APM, and AWS X-Ray all accept OTLP. The instrumentation decision is now separable from the backend decision, and the backend choice is a renewable one.

Where the wiring breaks

Even teams running OpenTelemetry, in production, often fail at one specific bridge: linking each log line back to the trace span it occurred inside.

The fix is twenty lines of code in any language with an OpenTelemetry SDK. Read the active span from the SDK at log time, attach trace_id and span_id to the structured log payload as additional fields. From that point forward, the log-aggregation tool and the APM are a single navigable surface — click a span in the trace view, see the logs for that span; open a log entry in the aggregator, jump to the trace that produced it.

The Node.js shape of it, with no extra dependencies beyond the OpenTelemetry SDK already in the application:

const { trace } = require('@opentelemetry/api');

function log(level, message, fields = {}) {
  const span = trace.getActiveSpan();
  const ctx = span ? span.spanContext() : {};

  console.log(JSON.stringify({
    level,
    message,
    trace_id: ctx.traceId || null,
    span_id: ctx.spanId || null,
    timestamp: new Date().toISOString(),
    ...fields,
  }));
}

// Use it like any structured logger.
log('error', 'Failed to reserve inventory', { item_id: 4821 });

Enter fullscreen mode Exit fullscreen mode

The same pattern transposes directly to Python (opentelemetry.trace.get_current_span()), Go (trace.SpanFromContext(ctx).SpanContext()), Java (Span.current().getSpanContext()), and every other OpenTelemetry SDK; the language-specific entry point changes, the structure does not. A wrapper like this routed through the team's existing logger (Pino, Winston, Bunyan, slog, Logback, etc.) means every log line emitted in a request inherits the trace-and-span IDs of whatever span is active at emission time.

The bridge is missing from a striking number of production deployments. Two screens, no link, the on-call engineer still grep-correlating by timestamp at 3am. The pattern is consistent enough across teams that it deserves to be called out as the single highest-leverage observability change a team can make: instrument once with OpenTelemetry, then add trace_id and span_id to every log line, and the entire observability surface becomes navigable.

The 3am scenario, replayed

Take the same incident from the opening, with traces wired. The trace view shows a POST /checkout request with 320ms total duration, breaking down into a 12ms hop from the API gateway to the order service, then a 301ms hop from order to inventory, where the inventory service's AcquireLock(item_id=4821) span shows a 300ms timeout. The full causal chain is visible on one screen. The trace_id on each log line in any log-aggregation tool is the same trace_id visible in the APM, so the engineer can pivot freely between the two surfaces.

Same incident, two debug tools, very different work envelope. The team that landed at the trace view in fifteen seconds is back in bed within the hour. The team without it is constructing the call graph by hand from log timestamps until daylight.

The economic point is the part most teams underestimate when they're prioritising the work. Incident MTTR is not just an availability metric — it is a developer-velocity metric, because it determines how much engineering time per week is spent in active incident response versus shipping features. A team running with traces wired has a bounded MTTR; the worst case is "look at the trace, find the slow span, fix it." A team without has an unbounded MTTR, because the worst case is "look at twenty log streams, hope someone wrote a useful field, give up and start adding println calls in a hotfix."

The minimum viable observability stack in 2026

The 2010s observability stack was complicated. In 2026 it isn't.

An OpenTelemetry SDK in the application emits OTLP traffic. An OpenTelemetry Collector — a small, stateless binary — receives that traffic and forwards it to whichever backend the team uses for storage and visualisation. The backend can be open-source self-hosted, commercial SaaS, or part of a wider observability suite. Switching backends is a Collector configuration change, not a re-instrumentation project.

The OTLP-compatible backend market in 2026, briefly:

Backend Type Hosting Notable strength
Jaeger Open source Self-host (Docker / Kubernetes / standalone) CNCF Graduated; OTLP-native receiver since 1.35; the reference implementation most other backends are read against
Grafana Tempo Open source Self-host or Grafana Cloud Object-storage-backed; tightly integrated with Loki (logs) + Prometheus (metrics) for a unified Grafana stack
SigNoz Open source Self-host or SigNoz Cloud OpenTelemetry-native end-to-end; ClickHouse-backed; combined logs/traces/metrics in one tool
Honeycomb Commercial SaaS Honeycomb Cloud High-cardinality query model; pioneered observability-as-debugging rather than as monitoring
Datadog APM Commercial SaaS Datadog Cloud Mature, widely deployed, expensive at high cardinality / high retention
New Relic Commercial SaaS New Relic Cloud Bundled into a broader APM / monitoring / RUM suite; consumption-based pricing
Splunk Observability Cloud Commercial SaaS Splunk Cloud Rebranded SignalFx; enterprise-oriented; integrates with the Splunk log / SIEM stack
Elastic APM Open source + commercial Self-host or Elastic Cloud Tight integration with Elasticsearch + Kibana; strong for teams already on the Elastic stack
AWS X-Ray Cloud-native AWS Native integration with AWS-only deployments; thin on cross-cloud or hybrid scenarios

Switching among these is a Collector configuration change. The application's instrumentation code does not move.

The application code carries one library and a handful of attribute-tagging calls in the spans the team cares about. Log-aggregation continues to use whatever the team already runs (Loki, Elastic, OpenSearch, Datadog Logs, etc.); the bridge is the trace_id field on every log line.

This is a much smaller commitment than the 2017 vendor-SDK pattern, and the lock-in surface is a fraction of what it was. Teams that haven't made the move are usually not held back by complexity; they're held back by the fact that the previous-generation advice columns are still in everyone's bookmarks, and the new pattern hasn't been internalised as the default.

What logs and traces actually answer

It is worth stating clearly, because the field's vocabulary has been muddy on this point for a decade. Logs answer what happened in this service at this moment. Traces answer where in the causal chain across services the failure occurred. Metrics answer how often, with what shape, over what window. Each of the three signals is necessary for a different class of question; none of them is sufficient on its own.

Signal Question it answers Unit of analysis Storage model Useful at 3am for
Logs What did this service observe at this moment? An event in one process Stream per service, time-ordered Reading the exact error text, stack trace, payload
Traces Where in the causal chain across services did the request go? A request, end to end Tree of spans per request Locating the slow or failing hop in a multi-service path
Metrics How often, with what shape, over what window? Counter / gauge / histogram, time-bucketed Time series Detecting that something is broken right now and characterising the pattern

The reason the three are routinely conflated is that the same word "observability" gets used for all of them, and the same vendor often sells all three as one product. They are still three different signals with three different jobs, and a deployment that has only one of them — usually logs, occasionally metrics — is missing two thirds of its debug surface.

The reason the 3am incident scenario is annoying is that the on-call engineer is asking a "where in the chain" question and being handed a "what at the moment" answer. The log line is technically correct and operationally useless, because it is the wrong signal for the question. Adding ten more log lines per service does not improve the situation; the additional log lines answer the same wrong question more loudly.

Wiring traces is not "doing observability properly." It is wiring the signal that the failure mode actually has a shape in. Once it is wired, the logs become useful again — not because the log lines got better, but because they now sit inside a structure that makes them addressable.

What the slow rollout actually looks like

There is a pattern in how teams adopt distributed tracing that is worth noting because it is so consistent.

A team starts with logs only. Logs are good enough until the deployment crosses some threshold, usually around four or five microservices, after which the log-correlation overhead becomes a daily friction. The team adds metrics, often Prometheus, and the metrics solve the "is something broken right now" question but do not help with "why is this specific request slow." The team adds tracing last, often after a particular incident has consumed enough engineering hours that someone has been given a quarter to fix it. The tracing rollout is not difficult; the SDK is mature, the Collector is small, the backends are interchangeable. The friction is organisational, not technical — the team has to allocate the time, agree on attribute conventions, and write the log-trace bridge.

After the rollout, every team that has done it reports the same observation: the marginal cost of the next incident drops by an order of magnitude. The incident that would have consumed an afternoon now takes ten minutes. The cost of the rollout pays back over the first two months.

Teams that haven't done it yet are paying that cost in incident hours, week over week, until they do.

The actual question

The question that matters in 2026 is not "should we do distributed tracing." The standardisation argument is over; OpenTelemetry has won the instrumentation layer, the W3C Trace Context spec has won the propagation layer, and the backend market has commoditised on the OTLP wire format. The question that matters is whether the team's logs are addressable from a trace, and whether the trace is addressable from a log line, in both directions, on every request.

If the answer is yes, the 3am incident has a different shape than the one in the opening. If the answer is no, the team is paying for that gap in MTTR every week, and the cost shows up in features that didn't ship because the on-call rotation was busy. Logs and traces are not in tension. They are two different signals, joined by twenty lines of code, and the team that has wired the join is the team that goes back to bed.