itay livni
02/08/2020, 4:35 PMKillworker
and dask-worker: error: unrecognized arguments: <tcp://127.0.0.5:8786>
with the second error always being any arguments I give the scheduler. Any thoughts on how to profile the DaskExecutor?Chris White
02/08/2020, 4:36 PMitay livni
02/08/2020, 4:38 PMexecutor = DaskExecutor(address="<tcp://127.0.0.5:8786>", debug=True)
or debug=Falseif make_nodes_executor=='dask':
from prefect.engine.executors import DaskExecutor
#client = Client()
executor = DaskExecutor(address="<tcp://127.0.0.5:8786>", debug=True) # Does not work
else:# make_nodes_executor=="local":
from prefect.engine.executors import LocalExecutor
executor = LocalExecutor()
Chris White
02/08/2020, 4:47 PMitay livni
02/08/2020, 5:21 PMFlow
within a LOOP
task
. That task
is triggered by another flow
run locally.Chris White
02/08/2020, 5:25 PMitay livni
02/08/2020, 5:27 PMChris White
02/08/2020, 5:29 PMitay livni
02/08/2020, 5:33 PMChris White
02/08/2020, 5:35 PMitay livni
02/08/2020, 5:39 PMmultiprocessing
or some other library/method?generate_knowledge_graph
is put together. It works but.. running it in the cloud or using storage does not. Basically anything that has a cloudpickle
fails. I need to learn how to chain and update flows.Chris White
02/08/2020, 7:46 PMitay livni
02/08/2020, 9:30 PM