@Marvin Where can I find an example of prefect logging configuration capturing logs from a different package?

An example of a Prefect logging configuration is not directly provided here, but you can see a simple Prefect flow which may include print statements as a form of logging within the task code. To capture logs from a different package using Prefect's logging system, you would configure Python's logging library in conjunction with Prefect's logger. To integrate Prefect with Python's logging system, you may follow these steps: 1. Import the

logging

module 2. Set up a logger using

logging.getLogger()

for the package you want to capture logs from 3. Set the desired log level (e.g.,

DEBUG

,

INFO

,

WARNING

,

ERROR

, or

CRITICAL

) using

logger.setLevel()

4. Integrate this logger with Prefect by adding it to Prefect's logger configurations if necessary Prefect uses Python's standard logging library, so if you configure logging at the start of your script, it will capture logs from the different packages used in your flow as long as those packages also use Python's standard logging system. I'll execute another tool to provide a Prefect-related example directly. I found a basic example of a Prefect flow where print statements are sent as INFO logs using the

log_prints=True

argument in the

@flow

decorator:

python
from prefect import flow

@flow(log_prints=True)
def my_workflow():
    print("I can call any python code here, including prefect tasks or flows")

if __name__ == "__main__":
    my_workflow()

You can find the complete example here: Basic Flow Example with Logging Moreover, I've found documentation that guides you on how to integrate Python's logging system with Prefect and how to customize it: - You can override any logging configuration by setting an environment variable or Prefect Profile setting with the syntax

PREFECT_LOGGING_[PATH]_[TO]_[KEY]

, for example,

PREFECT_LOGGING_LEVEL="DEBUG"

. - You can configure the "root" Python logger — the root logger receives logs from all loggers by default and is configured to output

WARNING

level logs to the console. - The document details how to adjust the log levels for specific handlers, for instance setting

PREFECT_LOGGING_HANDLERS_API_LEVEL=ERROR

to have only

ERROR

logs reported to the Prefect API. - Prefect's default logging configuration can be found in a

logging.yml

file, which you can customize by creating your own version with custom settings. For detailed information on how to work with logging in Prefect, you can refer to the Prefect Logging Documentation.