Hi everyone, when I register a flow with prefect-cloud/prefect-server I can see the task graph on prefect-server on the schematic tab (image shown below) of the UI. I would have thought that dask would calculate that graph. Is that true? If I'm using the dask executor is that graph then recalculated for each flow-run by the dask-scheduler? Or is the task graph calculated by dask at flow registration time and then cached somehow, so it doesn't need to be recalculated? If there is any documentation on this, please direct me there. Thank you!
03/30/2021, 3:27 PM
Hi @Adam Lewis -- we use Dask to execute the graph but we create the graph ourselves (ie we track tasks / edges in your flow). When the flow is registered, the graph is serialized into JSON and stored in the Cloud/Server backing database.