@Marvin I am using a task that uses

dask.delayed

for parallel computation. The logs inside this function don't make it to the prefect UI. Do you have any info on why that is?

The logs inside a function that uses

dask.delayed

might not be showing up in the Prefect UI due to how Dask handles logging. Dask has its own logging configuration which might not be compatible with Prefect's logging system. I recommend checking the logging configuration in your Dask setup and ensuring it's compatible with Prefect's logging system. If you're running your tasks in a Docker container, you might also want to check the Docker logs for any errors or issues. If you're still having trouble, you might want to consider posting a question on the Prefect Discourse or checking out the Prefect documentation for more information.

@Marvin I am getting this error:

UserWarning: Logger 'nameofmylogger' attempted to send logs to the API without a flow run id. The API log handler can only send logs within flow run contexts unless the flow run id is manually provided.

How can I manually provide the flow run id?

To manually provide a flow run id, you need to use the

context

of Prefect. The context is a dictionary that Prefect uses to store data that's globally accessible to all tasks within a flow run. The flow run id is stored in the context under the key

flow_run_id

. Here's an example of how you can access the flow run id from the context:

from prefect import context

flow_run_id = context.get('flow_run_id')

You can then use this

flow_run_id

in your logger. Please note that the context is only populated during a flow run, so if you try to access it outside of a flow run, it will return

None

. For more details, you can refer to the Prefect documentation.

@Marvin okay but what do I do with this flow run id? I already have it

Once you have the flow run id, you can use it to manually provide context to your logger. This is useful when you're logging from a location that doesn't automatically have context, like a separate thread or a Dask task. Here's an example of how you can use the flow run id with your logger:

from prefect import context
import logging

# Get the flow run id from the context
flow_run_id = context.get('flow_run_id')

# Create a logger
logger = logging.getLogger('nameofmylogger')

# Add the flow run id to the logger's extra context
logger = logger.bind(flow_run_id=flow_run_id)

# Now you can use the logger to log messages with the flow run id
<http://logger.info|logger.info>("This is a log message.")

In this example, the

bind

method is used to add extra context to the logger. This context will be included with every log message that's sent by the logger. This way, when you're viewing your logs in the Prefect UI, you'll be able to see which flow run each log message is associated with. For more information, you can refer to the logging section of the Prefect documentation.