https://prefect.io
Join Slack
Do Agents submit separate flow runs to the Executor in a depth-first or breadth-first pattern? I’m w...
# ask-community
j
Do Agents submit separate flow runs to the Executor in a depth-first or breadth-first pattern? I’m wondering if Prefect gives priority to task B of the 1st flow run (“1B”) over the task A of the 2nd flow run (“2A”).
Or is task 1B submitted to the Executor only after it’s Trigger is hit? I think this would lead to a race-condition for which task runs first (1B or 2A).
z
Hey @Jack Sundberg -- the Agent submits a flow then returns to querying for flows. They do not manage tasks at all.
Between Flow 1 and Flow 2 there is no coordination of task submission to an executor.
j
Gotcha, makes sense. For the core logic then (flow.run), is my 2nd statement above correct? Specifically a task is sent to the executor when the Trigger is hit (?)
z
Yes I believe so, otherwise we'd flood dask with a bunch of sleeping tasks.
We may submit a helper function to the executor when all of the upstream tasks for a task are completed that then checks the trigger and executes the real task or fails but I'm not sure if a case where this would affect you.
j
Okay, I just wanted to make sure flow.run() doesn't submit "delayed" tasks (like Dask.delayed).
and interesting... that doesn't affect me at the moment, but is good to know. Thanks!
z
flow.run
would submit dask delayed tasks if your executor is a 
DaskExecutor
j
yep! to clarify, I meant whether the full dask.delayed dag (so all flow tasks and logic) is submitted upfront to Executor, as opposed to only submitting tasks to the Executor when their Triggers indicate they are ready. Sounds like the latter is the actual Prefect case. Sorry for the confusion.
z
👍 all good
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by