How to Use Message Bus DLQs for Advanced Tail-Based Sampling
Author's Note: The EDA-based architecture introduced in this post is a proposed enterprise implementation pattern and is not part of the official OpenTelemetry Collector reference architecture.
The traditional two-tier OpenTelemetry (OTel) collector architecture can effectively address end-to-end tracing sampling challenges. However, it also introduces significant complexity and is difficult to implement in practice. One of the primary hurdles is the requirement for extremely powerful infrastructure—often necessitating high-performance servers with very large memory capacity—to handle traffic at tens of thousands of spans per second.
A key limitation of this traditional design lies in its point-to-point integration between the tier-1 and tier-2 collectors. This tight coupling forces the tier-2 collectors to independently manage resilience, scalability, and fault tolerance, which adds operational burden and limits flexibility.
In this section, we propose a modern alternative: replacing point-to-point communication with a messaging-based Pub/Sub integration. By decoupling the two tiers through a messaging layer, this approach...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
