Hi Chris, watched almost all of your videos on youtube. I have another problem
With running dask executor, dask-worker do not seem to recognize a module in its working directory.
Here is a minimal reproducable.
from test_custom import HelloWorld
helloworld_task = HelloWorld()withFlow("outside-task")as flow:
val = helloworld_task()
executor = DaskExecutor(address="<tcp://192.168.0.7:8786>")
flow_state = flow.run(executor=executor)
And here is what it is in test_custom.py
from prefect import Task
in local for debugging. All run from the same directory. And, if the flow is run without passing the
from that dir, it runs perfectly fine.
Oh, and sorry for not responding on your previous message @Chris WhiteNo, I wasn't looking for data dependency.
I want the below code to "taskIZE" in prefect.
for i in range(3):
for i in task_A():
return i + 1# The flow
And, I want the flow to do something like this:1. First run task_A
2. Whenever task_A yields something: pass it to task_B
3. Perellelly keep running task_A and task_B (when task_B has a single data from task_A).