Telemetry Tutorial#

scafctl emits all three OpenTelemetry signals — logs, traces, and metrics. By default, telemetry is silent (noop providers). When you point scafctl at an OTLP endpoint every signal is exported for backend analysis.

Overview#

Configuration#

Flags (per-invocation)#

# Point at a local OTel Collector
scafctl run solution -f solution.yaml \
  --otel-endpoint localhost:4317 \
  --otel-insecure          # disable TLS (development only)

# Override with environment variable instead
OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4317 \
  scafctl run solution -f solution.yaml

# Point at a local OTel Collector
scafctl run solution -f solution.yaml `
  --otel-endpoint localhost:4317 `
  --otel-insecure          # disable TLS (development only)

# Override with environment variable instead
$env:OTEL_EXPORTER_OTLP_ENDPOINT = 'localhost:4317'
scafctl run solution -f solution.yaml

Signals in Detail#

Logs#

Logs use the go-logr/logr interface throughout the codebase. The underlying sink is a multiSink that fans out to:

1. slog handler → stderr (text or JSON via --log-format)
2. otellogr bridge → OTel LoggerProvider → OTLP when --otel-endpoint is set

When no OTLP endpoint is configured, only the slog handler is active — no OTel log records are emitted to stderr. When an active span is in scope, the OTLP log record carries the trace_id and span_id automatically (correlation without any extra code).

Control verbosity:

scafctl run solution -f solution.yaml \
  --log-level debug \
  --log-format json \
  --otel-endpoint localhost:4317 \
  --otel-insecure

scafctl run solution -f solution.yaml `
  --log-level debug `
  --log-format json `
  --otel-endpoint localhost:4317 `
  --otel-insecure

See the Logging Tutorial for full flag reference.

Traces#

Spans are created at every major execution boundary:

All spans propagate W3C traceparent / tracestate headers on outbound HTTP requests via otelhttp.NewTransport, enabling distributed tracing when calling instrumented backends.

Local trace debugging#

Without --otel-endpoint, tracing is disabled (noop). To inspect traces locally, run a local collector such as otel-desktop-viewer or Jaeger and point scafctl at it:

# Start local Jaeger (see examples/telemetry/ for Docker Compose)
OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4317 \
  scafctl run solution -f solution.yaml --otel-insecure

# Start local Jaeger (see examples/telemetry/ for Docker Compose)
$env:OTEL_EXPORTER_OTLP_ENDPOINT = 'localhost:4317'
scafctl run solution -f solution.yaml --otel-insecure

Metrics#

Metrics are exported via the Prometheus bridge exporter. The MCP server exposes a /metrics endpoint (Prometheus scrape format). When --otel-endpoint is set, the same metrics are also pushed via OTLP gRPC at the default push interval.

Key metrics:

Running Locally with Jaeger#

The examples/telemetry/ directory contains a ready-to-use Docker Compose stack with Jaeger (all-in-one) and an OTel Collector.

Prerequisites#

• Docker and Docker Compose

Start the stack#

cd examples/telemetry
docker compose up -d

Services started:

Run a traced command#

OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4317 \
  scafctl run solution -f examples/actions/hello-world.yaml \
  --otel-insecure \
  --log-level debug

$env:OTEL_EXPORTER_OTLP_ENDPOINT = 'localhost:4317'
scafctl run solution -f examples/actions/hello-world.yaml `
  --otel-insecure `
  --log-level debug

View traces in Jaeger#

1. Open http://localhost:16686
2. Select service scafctl from the dropdown
3. Click Find Traces
4. Click any trace to see the waterfall of spans

Tear down#

cd examples/telemetry
docker compose down

Running with a Production Collector#

Point scafctl at your organisation’s OTLP endpoint (no --otel-insecure flag for TLS-enabled collectors):

scafctl run solution -f solution.yaml \
  --otel-endpoint otel-collector.example.com:4317 \
  --log-level info

scafctl run solution -f solution.yaml `
  --otel-endpoint otel-collector.example.com:4317 `
  --log-level info

Use the environment variable in CI/CD pipelines instead of flags:

# GitHub Actions
env:
  OTEL_EXPORTER_OTLP_ENDPOINT: otel-collector.example.com:4317
steps:
  - run: scafctl run solution -f solution.yaml --log-level info

Resource Attributes#

Every span, metric, and log record produced by scafctl includes:

Next Steps#

• Logging Tutorial — Log levels, formats, and file output
• MCP Server Tutorial — Host the MCP server and scrape /metrics
• examples/telemetry/README.md — Local Jaeger + Collector setup