https://prefect.io logo
c

Cristian Bob

10/30/2020, 11:51 PM
Hello! I am new into Prefect and for the last few days I've been testing its features. I feel enthusiastic about it and I really think that it can help us in defining robust and easy to understand workflows. I have one question regarding logging when running a flow using a
DaskExecutor
. In one of my tests I am running locally a flow using the
DaskExecutor
. The associated dask-scheduler as well as the dask-worker are started on my local computer. The problem is that the tasks' logged messages are being printed in the dask-worker's terminal instead of the terminal where I started the flow. Is there a way to get them printed in the latter terminal? I looked into the `DaskExecutor`'s documentation for an argument like
return_stdout
but I couldn't find anything relevant. Can you help me, please?
c

Chris White

10/30/2020, 11:56 PM
Hi @Cristian Bob and welcome! The situation you’re describing is a common frustration when first working with distributed computing environments. The creator of Dask, Matt Rocklin, has a great stackoverflow response on this exact topic here: https://stackoverflow.com/a/41476251/1617887 That being said, Prefect can help! If you run your workflows using a Prefect backend (either Server or Cloud), Prefect will collect these logs for you and display them all in one place in the UI.
c

Cristian Bob

11/16/2020, 10:22 PM
Hello, @Chris White! Thank you for your response!Yes, I think we will use Prefect server at some point in the future. For the moment we decided to start using Prefect core concepts to define our workflow since we have a logic for running tests on remote machines in place. Even though we've just started working on it, I think we have straight road to follow. BTW, very good documentation! 🙂
😄 1