Hi <@ULVA73B9P> what is the correct way to call `t...
# ask-marvin
b
Hi @Marvin what is the correct way to call
task.submit
where the task has keyword arguments, and I would like to
wait_for
a list of PrefectFutures?
m
thought for 207.9 seconds
In Prefect 3.x, pass your task’s keyword arguments as usual and add
wait_for
as a named argument to
submit
.
wait_for
can be a single
PrefectFuture
or a list of them. Example:
Copy code
from prefect import flow, task

@task
def transform(x, y=0, mode="fast"):
    return f"{mode}:{x + y}"

@flow
def main():
    a = some_setup.submit()
    b = other_task.submit(42)

    fut = transform.submit(
        10,           # positional arg for x
        y=5,          # keyword arg for the task
        mode="slow",  # another keyword arg
        wait_for=[a, b]  # list of PrefectFutures to wait on
    )

    return fut.result()
Notes: -
wait_for
is a parameter of
Task.submit
(and
Task.__call__
), not a task argument. Your task cannot declare parameters named
wait_for
or `return_state`—those names are reserved by Prefect. - If you pass a
PrefectFuture
as a task input (e.g.,
a=fut_a
), Prefect will automatically infer that dependency. Use
wait_for
when you need an ordering dependency that doesn’t come from using the upstream results as inputs. - You can also do the same with a direct task call (non-submitted):
transform(10, y=5, wait_for=[a, b])
if you want synchronous execution within the flow.
b
@Marvin With Prefect 2.x, what is the correct way to call
task.submit
where the task has keyword arguments, and I would like to
wait_for
a list of PrefectFutures?
m
thought for 217.9 seconds
In Prefect 2.x, pass your task’s keyword arguments as usual and add
wait_for
as a control kwarg to
submit
.
wait_for
takes an iterable of PrefectFutures and is not passed to your function. Signature (2.x):
Copy code
task.submit(*args, return_state=False, wait_for=None, **kwargs)
Example:
Copy code
from prefect import flow, task

@task
def process(name: str, count: int = 1, verbose: bool = False) -> str:
    return f"{name} x{count} (verbose={verbose})"

@flow
def my_flow():
    a = process.submit(name="alpha")
    b = process.submit(name="beta", count=2)

    # Provide your function kwargs + wait_for list of futures
    c = process.submit(name="gamma", count=3, verbose=True, wait_for=[a, b])

    return c.result()
Notes: -
wait_for
and
return_state
are reserved control kwargs consumed by Prefect; they are not passed into your task function. - You cannot define task parameters named
wait_for
or
return_state
(Prefect raises an error). Rename them if needed.
b
In Prefect 3, can I await an async task and also
wait_for
an iterable of PrefectFutures?