Prefect 2.0.4, dask, prefect UI displays logs from...
# ask-community
f
Prefect 2.0.4, dask, prefect UI displays logs from flows and subflows only, I have set loggers in task but those logs are not showing up in UI, I am using info logs only
1
c
Hi Faheem, Would it be feasible to test this in another venv using the latest (2.2.0) as of currently? Additionally, what does your code look like to log those tasks? I have seen reports in the past where tasks do eventually get logged, but it’s not immediately - that is, the flows and subflows log in real-time, but tasks (because they are distributed exection in dask) are not quite real-time, as the results are batch written out
👍 1
f
@Christopher Boyd thanks, I have tried latest 2.2.0 ran into some issues had to roll back, https://prefect-community.slack.com/archives/CL09KU1K7/p1661759411186159 asked in this thread.
With the logging issue I think, as u said it would be because of the task being distruibet on dask cluster. what do u suggest the best approach would for logging in the tasks?
c
By their nature, they are ephemeral, so have to be logged in the task themselves. I would ensure you are getting getting a new task logger context (in the task itself) and logging the output during that task. This might be slightly asynchronous (not real-time logging) as I mentioned, because the tasks and dask pods will queue writes to the batch writers,
If you can post a snippet of how you have your logging setup for the tasks?