@Marvin I have set

export PREFECT_LOGGING_EXTRA_LOGGERS=mypackage.mymodule.myjob

, but I get this error:

/home/ubuntu/Projects/Prefect/myproject/mypackage.mymodule.myjob.py:26: UserWarning: Logger 'mypackage.mymodule.myjob' attempted to send logs to the API without a flow run id. The API log handler can only send logs within flow run contexts unless the flow run id is manually provided. Set PREFECT_LOGGING_TO_API_WHEN_MISSING_FLOW=ignore to suppress this warning.

How do I manually provide the flow run id? Can it be set in the environment?

Ah, the joys of logging configuration! 👀 Let's find a way to manually provide that flow run ID to prevent those warnings from cluttering up your logs. First, let me know which version of Prefect you're working with - this will guide the search to get you the most relevant info.

3.0.2

From what I've found in both Prefect 2.x and 3.x documentation, here's how you can provide the flow run ID when logging: ### Prefect 2.x 1. Custom Loggers: You can create custom loggers that are aware of the flow run context by using Prefect's

get_run_logger

. Prefect automatically provides the flow run ID when you use this method:

python
   from prefect import flow, get_run_logger
   
   @flow(name="log-example-flow")
   def logger_flow():
       logger = get_run_logger()
       <http://logger.info|logger.info>("INFO level log message.")

2. Environment Variables: While there's no direct mention of setting the flow run ID via environment variables, you can configure logging settings by creating your own

logging.yml

and specifying it with

PREFECT_LOGGING_SETTINGS_PATH

. ### Prefect 3.x 1. Flow/Task Run Loggers: Similar to 2.x, use the

get_run_logger

function to get a logger that has access to `flow_run_id`:

python
   from prefect import flow, get_run_logger
   
   @flow
   def my_flow():
       logger = get_run_logger()
       <http://logger.info|logger.info>("This is a log message.")

2. Logging Configuration: You can adjust log formatting to include the

flow_run_id

by setting environment variables such as:

plaintext
   PREFECT_LOGGING_FORMATTERS_STANDARD_FLOW_RUN_FMT="%(asctime)s.%(msecs)03d | %(levelname)-7s | %(flow_run_id)s - %(message)s"

For detailed steps on logging configuration and setting these variables, refer to these: - Prefect 2.x Custom Logging - Prefect 3.x Logging Let me know if this helps or if you need more details!

Unfortunately the function I need output for is deep inside another package. The package function I call from my task eventually calls down to the function I want standard (non-prefect run logger) logging output from. I don't want to capture from the root logger if I can help it because this clutters the output with many libraries like httpx.

### Filtering Logs from Specific Packages in Prefect Both in Prefect 2.x and 3.x, you can filter which packages Prefect captures from the root logger using the

PREFECT_LOGGING_EXTRA_LOGGERS

setting. #### Setting Environment Variables 1. Specify Loggers to Include You can specify which libraries to capture logs from by setting the

PREFECT_LOGGING_EXTRA_LOGGERS

environment variable:

export PREFECT_LOGGING_EXTRA_LOGGERS=dask,scipy

2. Set Logging Levels You can also control the logging level for different loggers including the root logger:

plaintext
   export PREFECT_LOGGING_ROOT_LEVEL="WARNING"

#### Customizing Logging Configuration File You can create a custom

logging.yml

file and specify which loggers should be included or excluded and define their levels. Use

PREFECT_LOGGING_SETTINGS_PATH

to point to your custom configuration. #### Example Logging Configuration in yml

yaml
version: 1
disable_existing_loggers: false
loggers:
  prefect:
    handlers: [console]
    level: INFO
  dask:
    handlers: [console]
    level: DEBUG
root:
  handlers: [console]
  level: WARNING

For more details, you can refer to the Prefect documentation: - Prefect 2.x Logging Configuration - Prefect 3.x Logging If you want further fine-grained control, modify the

logging.yml

to disable specific loggers or set their propagation settings. For instance:

yaml
loggers:
  dask:
    propagate: False
    level: WARNING

Always ensure to restart your service after updating environment variables. Let me know if you need more help!