Broder Peters
02/06/2023, 11:38 AMprefect deployment build my_flow.py:do_parallel_stuff -n docker -ib docker-container/docker -sb s3/some-bucket
@flow(task_runner=DaskTaskRunner)
def do_parallel_stuff():
for x in y:
result = do_heavy_stuff_in_own_environment.submit(x)
@task()
def do_heavy_stuff_in_own_environment(x: str):
# Does heavy stuff
Thanks in advance!talat
02/06/2023, 4:55 PMrun_deployment
on the child-flow X number of times. since the child-flow is a separate deployment, it can have its own infrastructure and storage to run on. hthBroder Peters
02/07/2023, 6:49 AMtalat
02/07/2023, 8:24 AM