Today's digest

The OpenTelemetry Collector is a vendor-agnostic binary that receives, processes, and exports telemetry data (traces, metrics, logs). It is written in Go, licensed under Apache-2.0, and has a large community with 7k stars and active maintenance. The core component provides a pipeline architecture with receivers, processors, and exporters, and supports OTLP as well as many other formats. For Truebot, which currently has a noop telemetry layer (internal/telemetry), the Collector offers a production-grade path to obtain real observability. For Truebot specifically, there are three concrete plug points where it earns its place. Listed in increasing ambition: 1. Replace the noop telemetry in internal/telemetry with the OpenTelemetry Go SDK. This involves initializing a TracerProvider and MeterProvider that export via OTLP to a local Collector instance. The existing internal/telemetry package is a placeholder; you would add a new file, say internal/telemetry/otel.go, that configures OTLP exporters and registers them during application bootstrap (in internal/app). This gives you distributed tracing across agent ops and metrics on request latency, memory usage, and channel throughput with minimal code changes. Estimated time: 2-3 days for integration with existing logging. 2. Run the OpenTelemetry Collector as a sidecar process alongside agentd to receive OTLP data and handle backpressure, batching, and retries. In your docker-compose.yml (or as a separate systemd service for local mode), add a collector container configured with a YAML file (internal/telemetry/collector-config.yaml) that uses the OTLP receiver and exports to stdout or a file for now. This separates concerns: the application only emits telemetry, and the Collector manages delivery. You can then add exporters for Prometheus (for metrics) and Jaeger (for traces) without touching app code. Estimated time: 1-2 days for config and deployment. 3. Leverage the Collector's built-in processors for resource detection and sampling. In the collector config, add a batch processor to group spans/metrics before export, a memory_limiter to prevent OOM, and an attributes processor to tag telemetry with environment, version, or channel name from Truebot's gateway. These processors run in the Collector process and do not require changes to the application. This is the final step to production readiness. Estimated time: half a day to tune config. The smallest viable first slice is plug point 1: instrument the noop telemetry with the OTel Go SDK and point it at a local Collector that writes to stdout. This requires adding the OTel dependencies, creating an init function in internal/telemetry, and updating the bootstrap in internal/app/config.go. It builds on nothing else and can be done in 2-3 days. Plug points 2 and 3 are additive; they depend on having the SDK emitting OTLP so the Collector has data to process. Start with the SDK integration, then add the Collector sidecar, then tune processors.