Ben Welsh
01/28/2022, 2:42 PMKevin Kho
myflow.py
but register it with some registration script import flow from myflow
and then attach your configurations
import flow from myflow
flow.run_config = ...
flow.executor = ...
flow.storage = ..
flow.register(...)
This is because all of the other guys are stored (RunConfig, Storage) on the Prefect database, but the executor is not part of the serialized Flow since it can have sensitive info (think Dask cluster address).
So the executor is pulled from the Flow file that is stored so it needs to be defined there.
Running the flow with debug level logs also lets you know what executor is being used.Kevin Kho
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by