Short answer: Yes. In Prefect 3.x, the engine is instrumented with OpenTelemetry and will emit spans for every flow and task run. You just need an exporter configured to actually see them. What to do based on your setup: - Prefect Cloud: Orchestration telemetry is on by default. As long as your runs are connected to Cloud and

PREFECT_CLOUD_ENABLE_ORCHESTRATION_TELEMETRY

is true (default), spans are created and exported to Cloud automatically. - Setting reference: Prefect 3 settings (see

PREFECT_CLOUD_ENABLE_ORCHESTRATION_TELEMETRY

) - Your own OTEL backend (Jaeger, Datadog, OTEL Collector, etc.): Configure standard OTEL environment variables in the environment where your flows run. For example:

OTEL_TRACES_EXPORTER=otlp
  OTEL_EXPORTER_OTLP_ENDPOINT=<http://otel-collector:4317>
  OTEL_EXPORTER_OTLP_PROTOCOL=grpc
  OTEL_SERVICE_NAME=prefect

With those set, the built-in spans created by Prefect’s engine for every flow/task will be exported to your collector. Optional: add your own custom spans inside flows/tasks - Prefect propagates the active trace context, so custom spans will nest under the flow/task spans:

from opentelemetry import trace
  from prefect import flow, task

  tracer = trace.get_tracer("my-app")

  @task
  def do_work():
      with tracer.start_as_current_span("custom-step"):
          # your logic here
          return "done"

  @flow
  def my_flow():
      return do_work()

  if __name__ == "__main__":
      my_flow()

Helpful references: - Run telemetry API: prefect.telemetry.run_telemetry - Telemetry bootstrap/instrumentation (exporter setup): - prefect.telemetry.bootstrap - prefect.telemetry.instrumentation - Prefect 3.2 notes introducing orchestration telemetry: v3.2 release notes Quick clarifier so I can tailor this: - Are you using Prefect Cloud or OSS? - Where do you want the spans to be exported (Cloud, Jaeger, Datadog, another OTEL backend)? - How are your flows executed (local, Docker/K8s work pools)? I can share the exact env var configuration for that environment.

Perfect — here’s how to get automatic spans from every flow/task to your OTEL Collector when running Prefect OSS on Kubernetes work pools. What Prefect does for you - Prefect 3.x automatically creates OpenTelemetry spans for every flow and task run. You don’t need to change your flow code. - You just need to: (1) have the OTEL SDK/exporter available in your flow run image and (2) set OTEL env vars so the spans are exported to your collector. Step-by-step for Kubernetes work pools 1) Use an image that includes the OTEL SDK - Build a small image for your flow runs:

FROM prefecthq/prefect:3-python3.11
  RUN pip install --no-cache-dir opentelemetry-sdk opentelemetry-exporter-otlp

Push it to your registry and note the image name (e.g.,

your-registry/prefect-otel:latest

). 2) Configure your K8s work pool’s base job template with the image + OTEL env vars - Grab the default template:

prefect work-pool get-default-base-job-template --type kubernetes > k8s-base.json

- Edit

k8s-base.json

to: - Set the

image

to your custom image - Add env vars for your collector. A common configuration looks like:

{
  "variables": {
    "image": "your-registry/prefect-otel:latest",
    "env": [
      {
        "name": "OTEL_TRACES_EXPORTER",
        "value": "otlp"
      },
      {
        "name": "OTEL_EXPORTER_OTLP_ENDPOINT",
        "value": "<http://otel-collector.default.svc.cluster.local:4317>"
      },
      {
        "name": "OTEL_EXPORTER_OTLP_PROTOCOL",
        "value": "grpc"
      },
      {
        "name": "OTEL_SERVICE_NAME",
        "value": "prefect"
      },
      {
        "name": "OTEL_RESOURCE_ATTRIBUTES",
        "value": "service.namespace=data-eng,deployment.environment=prod"
      }
    ]
  }
}

Notes: - If your collector exposes HTTP/4318 instead of gRPC/4317, use: -

OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

-

OTEL_EXPORTER_OTLP_ENDPOINT=<http://otel-collector.default.svc.cluster.local:4318>

- For auth/TLS to a remote collector, add

OTEL_EXPORTER_OTLP_HEADERS

and/or

OTEL_EXPORTER_OTLP_CERTIFICATE

as needed. - Apply the template to your pool:

prefect work-pool update "<your-pool-name>" --base-job-template k8s-base.json

3) Deploy and run - No code changes are required; Prefect’s engine will emit spans for each flow/task and your OTEL Collector will receive them. Optional: add custom spans inside tasks - If you want additional granularity, you can nest your own spans under Prefect’s:

from opentelemetry import trace
  from prefect import flow, task

  tracer = trace.get_tracer("my-app")

  @task
  def do_work():
      with tracer.start_as_current_span("custom-step"):
          # your logic here
          return "done"

  @flow
  def my_flow():
      return do_work()

  if __name__ == "__main__":
      my_flow()

Troubleshooting checklist - No traces showing up? - Confirm your flow-run pods have the OTEL env vars you set. - Confirm the image includes

opentelemetry-sdk

and

opentelemetry-exporter-otlp

. - Verify network access to the collector service name/port in the same namespace. - Enable debug logs from the OTEL SDK: add

OTEL_LOG_LEVEL=debug

. - Consider sampling controls: -

OTEL_TRACES_SAMPLER=parentbased_traceidratio

-

OTEL_TRACES_SAMPLER_ARG=0.1

(10% sampling) References - Prefect telemetry modules and API: - prefect.telemetry.run_telemetry - prefect.telemetry.bootstrap - prefect.telemetry.instrumentation - Prefect 3 settings: settings reference If you want, share your current base job template and the collector’s endpoint/protocol, and I’ll tailor the exact JSON you need.

Thanks — that symptom usually means your OpenTelemetry configuration is exporting traces (and maybe some non-Prefect logs), but Prefect’s run logger isn’t being sent to your OTEL logs pipeline. What’s likely happening - Prefect automatically emits spans, but logs are a separate signal. Having

OTEL_TRACES_EXPORTER=otlp

won’t export Python logs by itself. -

get_run_logger()

uses Python logging under the hood, but those records won’t reach OTEL unless: 1) an OTEL LoggingHandler is attached, and 2) your OTEL Collector has a logs pipeline enabled, and 3) the handler/log levels allow INFO through. - In many setups the Prefect logger doesn’t propagate to the root logger, so attaching an OTEL handler only to the root logger won’t capture

get_run_logger()

messages. How to fix (K8s work pools) 1) Ensure OTEL logs are enabled in your flow-run image - Your image needs the logs pieces:

pip install opentelemetry-sdk opentelemetry-exporter-otlp

(You already have these for traces, but confirm versions that include logs support.) 2) Add OTEL env vars for logs to the work pool’s base job template - In the same place you added spans config, also add:

OTEL_LOGS_EXPORTER=otlp
  OTEL_EXPORTER_OTLP_ENDPOINT=<http://otel-collector.default.svc.cluster.local:4317>
  OTEL_EXPORTER_OTLP_PROTOCOL=grpc
  OTEL_SERVICE_NAME=prefect
  OTEL_PYTHON_LOG_CORRELATION=true

If your collector uses HTTP/4318, switch protocol to

http/protobuf

and the endpoint accordingly. 3) Attach an OTEL logging handler to Prefect’s loggers - Do this early in your flow code (e.g., import a small module before creating the flow):

# otel_logging.py
  import logging

  from opentelemetry._logs import set_logger_provider
  from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
  from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
  from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter

  def enable_otel_logging(level=<http://logging.INFO|logging.INFO>):
      provider = LoggerProvider()
      set_logger_provider(provider)
      exporter = OTLPLogExporter()  # uses OTEL_* env vars
      provider.add_log_record_processor(BatchLogRecordProcessor(exporter))
      handler = LoggingHandler(level=level, logger_provider=provider)

      # Attach to Prefect loggers commonly used by get_run_logger()
      for name in ("prefect", "prefect.flow_runs", "prefect.tasks"):
          l = logging.getLogger(name)
          l.addHandler(handler)
          # Allow propagation in case you also attach root-level handlers
          l.propagate = True

  enable_otel_logging()

4) Confirm your OTEL Collector has a logs pipeline - If your collector only has a traces pipeline, logs will be dropped. Minimal example:

receivers:
    otlp:
      protocols:
        grpc:
        http:
  processors:
    batch: {}
  exporters:
    logging: {}  # or your backend exporter
  service:
    pipelines:
      logs:
        receivers: [otlp]
        processors: [batch]
        exporters: [logging]

5) Verify levels and sampling - Make sure your handler and logger level are INFO or lower (so

<http://logger.info|logger.info>("Lala")

passes). - For debugging, you can add

OTEL_LOG_LEVEL=debug

to see OTEL SDK activity in the pod logs. After this, you should see: - Your flow/task spans in your OTEL backend (unchanged). - Your

get_run_logger()

messages exported as OTEL logs and correlated with the active trace (thanks to

OTEL_PYTHON_LOG_CORRELATION=true

). References - Prefect logging customization: Advanced logging customization - Prefect telemetry modules (API reference): - prefect.telemetry.run_telemetry - prefect.telemetry.bootstrap

- prefect.telemetry.instrumentation - prefect.telemetry.logging If you can share your current base job template env vars and (sanitized) collector pipeline config, I can spot-check and suggest exact changes.