Yury Cheremushkin
08/08/2022, 12:14 PM.set_upstream()
method in 1.0. But it looks like there are no more such method. I.e. there are two tasks: loading data into some BigQuery table and then merging it with another table. Obviously these tasks need to be run in exact order. But there are no need to pass any results from the first task to the second, so it wasn’t obvious for Prefect 1.0 that there is some kind of order. That’s why I used .set_upstream()
. What should i do now?
And the third question is about .map()
for tasks. So, now i should just use standard pythonic map? Will it be parallelized in the same manner as it was in Prefect 1.0 with Concurrent/Parallel task runner?Anna Geller
08/08/2022, 12:34 PMFailed
task run:
from prefect import task
@task
def signal_task(message):
if message == 'stop_immediately!':
raise RuntimeError(message='Got a signal to end the task run!')
Alternatively, your flow may return a specific state 3:
from prefect import task
from prefect.orion.schemas.states import Failed
@task
def signal_task(message):
if message == 'stop_immediately!':
return Failed(message='Stopping the task run immediately!')
Yury Cheremushkin
08/08/2022, 2:52 PMAnna Geller
08/08/2022, 2:58 PMMichael Law
08/29/2022, 4:40 PMAnna Geller
08/30/2022, 11:16 AM