@Marvin can I just use an existing logger that I was already using, and not the one provided by prefect. can I do that

Yes, you can use an existing logger with Prefect instead of the one provided by Prefect. Prefect automatically configures its own logging for flows and tasks, but you are free to use your custom logger for logging specific events or information important to your workflow. For more details, you can refer to the Prefect logging documentation.

@Marvin

logger = logging.getLogger("hi")

In [11]: @flow
    ...: def my_flow():
    ...:     my_task()
    ...:

In [12]:

In [12]: @task
    ...: def my_task():
    ...:     logger.warning("inside the flow")

When I run this, I see the logs

00:29:51.630 | INFO    | prefect.engine - Created flow run 'mini-falcon' for flow 'my-flow'
00:29:54.381 | INFO    | Flow run 'mini-falcon' - Created task run 'my_task-0' for task 'my_task'
00:29:54.383 | INFO    | Flow run 'mini-falcon' - Executing 'my_task-0' immediately...
00:29:55.194 | WARNING | hi - inside the flow
00:29:56.166 | INFO    | Task run 'my_task-0' - Finished in state Completed()
00:29:56.410 | INFO    | Flow run 'mini-falcon' - Finished in state Completed('All states completed.')

and it seems the logger's configuration is modified somehow, within the flow or the task

It seems your logger's configuration is being influenced by Prefect's default logging setup. Prefect automatically configures logging for flows and tasks, which might affect your custom logger unless you explicitly configure it otherwise. Prefect's logging can be customized by: - Modifying the root logger settings. - Using environment variables like

PREFECT_LOGGING_ROOT_LEVEL

to change logging levels. - Providing your own

logging.yml

configuration file in your

PREFECT_HOME

directory to completely overhaul Prefect's logging. For more details, refer to the Prefect logging documentation.

@Marvin how can I turn off that prefect's logging set up

To turn off or modify Prefect's logging setup, you can provide your own logging configuration. Here's how: 1. Override the Root Logger: You can configure the root logger to change its behavior. By default, it outputs

WARNING

level logs to the console. You can modify this by setting the

PREFECT_LOGGING_ROOT_LEVEL

environment variable. 2. Custom Logging Configuration: Create your own

logging.yml

file in your

PREFECT_HOME

directory. This allows you to completely overhaul Prefect's logging configuration. For more detailed instructions, you can refer to the Prefect logging documentation.