So I've been diving deep into the observability rabbit hole lately and hit a wall with this: distributed tracing feels like the holy grail for microservices debugging, but everyone also sweeps traditional centralized logging under the rug. When do you actually need distributed tracing over logs? Is it just for latency hotspots, or are there scenarios where logs leave you blind? Also, what's the real overhead cost in production environments? Looking for war stories or architecture decisions that made the difference between a 3am firefight and a quiet Monday morning.
Distributed tracing vs. traditional logging: where's the catch?
👁️ 4 views💬 1 replies❤️ 0 likes
1 Replies
I was also eagerly setting up Jaeger + OpenTelemetry, only to realize after the first month that trace IDs were being reused due to container restarts, and it took two full days of debugging to uncover that all the metrics were fake. "This pitfall is truly impossible to avoid in large-scale auto-scaling scenarios."
The real game-changer comes in **client-side cross-process + cross-language** business scenarios: like a mini-program requesting microservices A→B→C→an external payment gateway. Traditional logs with correlation IDs can still match things up, but Redis performance dips or Kafka consumption delays just disappear into the void in logs. Only traces can tell me which hop in the chain caused that 98 ms delay. That's when I realized logs are like "only seeing the fire in your own house but not where it started."
Resource overhead is definitely not something to ignore: each trace adds ~4–8 KB payload, and on a high-concurrency service with 5k QPS, that's an extra 1.2–2 TB of network & storage daily. Now we only store kernel-level traces (like syscalls, gRPC interceptors) separately from business traces—5 days vs. 30 days—and the cost is almost halved. My advice? If you can't even handle day-2 incidents, get logging sorted first before overhauling your entire observability stack.