Hi all, is there a way I can specify a task to run only once within a flow? I have noticed some of m...

Izu

06/13/2022, 11:28 AM

Hi all, is there a way I can specify a task to run only once within a flow? I have noticed some of my tasks run concurrently, this is an issue when I am writing to a table

✅ 1

Anna Geller

06/13/2022, 11:31 AM

Sure, this is what concurrency limits are for

Anna Geller

06/13/2022, 11:32 AM

you can set a limit of 1 to only allow one active flow run of a given flow

Anna Geller

06/13/2022, 11:32 AM

(or one active task run with a given tag)

Anna Geller

06/13/2022, 11:33 AM

and to ensure your tasks run sequentially, you can use SequentialTaskRunner in Prefect 2.0 and LocalExecutor in Prefect 1.0

Izu

06/13/2022, 11:34 AM

Thank you Anna, is there documentation I could use to look this up? https://prefect-community.slack.com/archives/CL09KU1K7/p1655119965466709?thread_ts=1655119693.324589&cid=CL09KU1K7

Anna Geller

06/13/2022, 11:40 AM

it depends - are you asking for Prefect 2.0 or Prefect 1.0?

Anna Geller

06/13/2022, 11:40 AM

1.0 https://docs.prefect.io/ 2.0 https://orion-docs.prefect.io/ and search for concurrency limits

Anna Geller

06/13/2022, 11:41 AM

also https://discourse.prefect.io/tag/concurrency-limits

9 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.