DEV Community

Shubham Chavan
Shubham Chavan

Posted on

🚀 Getting Started with OpenTelemetry for Python Microservices on Kubernetes

#devops #opentelemetry #monitoring

Here’s a clean, step-by-step guide to help you get up and running with OpenTelemetry in a real-world, containerized setup ideal for Python-based services running in Kubernetes.

🆚 Auto-Instrumentation vs Manual-Instrumentation in OpenTelemetry

Feature Auto-Instrumentation Manual Instrumentation
Definition Automatically instruments supported libraries and frameworks using OpenTelemetry wrappers. Requires you to explicitly define spans in code using the OpenTelemetry API.
Setup Minimal changes — just install dependencies and run via opentelemetry-instrument CLI. Requires importing OTEL SDK and wrapping logic with spans manually.
Use Cases Quick observability setup, ideal for supported web frameworks like Flask, Django, FastAPI, SQLAlchemy, etc. Fine-grained control — custom logic, background jobs, async flows, or business-critical code.
Example opentelemetry-instrument python app.py python with tracer.start_as_current_span("process-order"): do_work()
Pros Easy, fast, low effort. Captures most useful traces out of the box. Complete control over what is traced, how it’s labeled, and where context is passed.
Cons Limited to what the SDK supports; may miss custom logic or async paths. Requires effort, code changes, and understanding of OTEL trace context.

🧩 Architecture Overview

We’ll use:

  1. OpenTelemetry SDK in a Python app

  2. OTEL Collector Agent (sidecar or DaemonSet on app clusters)

  3. OTEL Collector Gateway (centralized, on a GitOps or management cluster)

  4. SigNoz as the visualization backend (optional – you can swap with Jaeger, Datadog, etc.)

📦 Step 1: Instrument Your Python Application

Install OpenTelemetry dependencies:

pip install opentelemetry-sdk \
            opentelemetry-exporter-otlp \
            opentelemetry-instrumentation \
            opentelemetry-instrumentation-flask  # or your framework
Enter fullscreen mode Exit fullscreen mode

Initialize OTEL in your code:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)

otlp_exporter = OTLPSpanExporter(endpoint="http://otel-collector:4318/v1/traces")
span_processor = BatchSpanProcessor(otlp_exporter)
trace.get_tracer_provider().add_span_processor(span_processor)
Enter fullscreen mode Exit fullscreen mode

Instrument libraries:

opentelemetry-instrument python app.py
Enter fullscreen mode Exit fullscreen mode

🏗️ Step 2: Deploy OpenTelemetry Collector Agent

Use the official Helm chart or a custom otel-agent.yaml.

Basic example (DaemonSet mode):

receivers:
  otlp:
    protocols:
      http:
      grpc:

exporters:
  otlp:
    endpoint: <gateway-collector>:4317
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlp]
Enter fullscreen mode Exit fullscreen mode

🌐 Step 3: Deploy OTEL Collector Gateway

This collector aggregates all telemetry. Example config:

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  signoz:
    endpoint: http://signoz-otel-collector:4317

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [signoz]
Enter fullscreen mode Exit fullscreen mode

📊 Step 4: Install SigNoz or Jaeger as Your Observability Backend

helm repo add signoz https://charts.signoz.io
helm install signoz signoz/signoz

🛠 Tips

  • Use resource attributes to tag services (service.name, env, etc.).
  • Add custom spans around critical code blocks.
  • Use batch processors to reduce export overhead.

Top comments (0)