with Prefect 2 tasks, if it’s possible that is? I’d like to be able to skip a failed task and let downstream tasks proceed but it seems like a Failed task kills the entire flow. Or maybe I’m wrong?
a
Andrew Huang
01/09/2023, 7:01 PM
I think you want allow_failure, but I think try/except also should work
z
Zanie
01/09/2023, 7:02 PM
Copy code
from prefect import task, flow, allow_failure
@task
def fail():
raise ValueError()
@task
def succeed(x):
return x
@flow
def example():
try:
fail()
except ValueError:
print("Ah it failed")
print("All good!")
state = fail(return_state=True)
if state.is_failed():
print("Ah it failed")
print("All good")
future = fail.submit()
try:
future.result()
except ValueError:
print("Ah it failed")
print("All good!")
return succeed(allow_failure(future))
example()
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.