Skip to content

Copilot Chat otel.captureContent:false not honored on spans — chat spans carry full prompts/system instructions/responses (copilot-chat 0.57.0 in core) #326254

Description

@Seacrest2023

Type: Bug

github.copilot.chat.otel.captureContent: false is not honored on the span path — chat spans carry full prompts, system instructions, and responses

Summary

With OTel emission enabled and captureContent explicitly false (the default), span attributes still include complete conversation content. The log and metric paths honor the setting; the span path does not. The setting's description says: "Capture input/output messages, system instructions, and tool definitions in OTel telemetry. Contains potentially sensitive data." — so users reasonably rely on false to keep code and prompts out of their telemetry backend.

Environment

  • Copilot Chat 0.57.0 (built into VS Code core)
  • VS Code stable, Windows 11 (10.0.26200)
  • Settings:
    "github.copilot.chat.otel.enabled": true,
    "github.copilot.chat.otel.exporterType": "otlp-grpc",
    "github.copilot.chat.otel.protocol": "grpc",
    "github.copilot.chat.otel.otlpEndpoint": "http://127.0.0.1:4317",
    "github.copilot.chat.otel.captureContent": false
  • No COPILOT_OTEL_CAPTURE_CONTENT environment variable set (no enterprise policy either).

Repro

  1. Configure as above, pointing at any OTLP collector (we used otelcol-contrib 0.156.0 with a file exporter).
  2. Reload the window; run any chat request.
  3. Inspect the exported chat spans.

Observed

Every chat span carries, with captureContent: false:

  • copilot_chat.user_request — the complete user prompt (6,505 chars in our repro)
  • gen_ai.system_instructions — full system instructions (1,162 chars)
  • gen_ai.input.messages — the complete message array
  • gen_ai.output.messages — the full assistant response
  • gen_ai.tool.call.arguments, gen_ai.tool.call.result, gen_ai.tool.definitions

Log records and metrics in the same capture correctly contain metadata only (token counts, model, timing) — the gating appears to be applied on those paths but missing on the span-building path.

Expected

With captureContent: false, span attributes contain no message content, no system instructions, no tool arguments/results — matching the log/metric behavior and the setting's description.

Impact

Anyone who enabled OTel spans while relying on captureContent: false is shipping full conversation content (which can include proprietary source code) to their telemetry backend without knowing it. For our part we've had to exclude span capture from all customer-adjacent environments until this is fixed — which is a pity, because the span path is the only place cache token detail (gen_ai.usage.cache_read.input_tokens) and copilot_chat.copilot_usage_nano_aiu are emitted, and those are exactly the attributes cost-observability setups want.

Happy to provide sanitized span samples on request.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions