Ivan Ksaver Šušnjara
04/22/2021, 1:37 PMRuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
What is more intriguing, some flows continue and complete nonetheless, while other require manual KeyboardInterrupt (ctrl+c), and then they start executing their respctive tasks. Dask provides solution for the issue (https://github.com/dask/distributed/issues/2520), however I don't see a workaround for this in Prefect. Problem does not appear when "threads" are employed instead of "processes" with LocalDaskExecutor's scheduler argument.
I'm using Prefect version 0.14.16 with Python 3.8.7.
Has anyone faced the same issue? Thanks in advance!Kevin Kho
04/22/2021, 1:41 PMIvan Ksaver Šušnjara
04/22/2021, 1:41 PMKevin Kho
04/22/2021, 1:44 PMflow.executor = LocalDaskExecutor()
under the if __name____ == __main___:_
blockIvan Ksaver Šušnjara
04/22/2021, 1:45 PMif __name__ == "__main__"
block does not help here unfortunatelyKevin Kho
04/22/2021, 2:17 PMif __name__ == "__main__"
?Ivan Ksaver Šušnjara
04/22/2021, 2:20 PMNameError: name 'flow' is not defined
. I'm using context manager (with statement in python) to define flow, and trying to run it from the same script, calling flow.run() methodKevin Kho
04/22/2021, 2:40 PMfrom prefect import Flow, task
import prefect
import logging
from prefect.executors import LocalDaskExecutor
@task
def mytask():
logger = prefect.context.get('logger')
<http://logger.info|logger.info>('hello')
if __name__ == "__main__":
with Flow('test') as flow:
mytask()
flow.executor = LocalDaskExecutor()
flow.run()
if __name__ == "__main__":
flow.run()
Ivan Ksaver Šušnjara
04/23/2021, 6:28 AMMichael Hadorn
06/11/2021, 12:47 PM