FuETL
11/21/2022, 11:05 AMFAIL
, PrefectStateSignal
, PrefectSignal
) inherit from BaseException
instead of Exception
?try:
...
except Exception as err:
pass
and raising fail signal it was not being catched.Anna Geller
11/21/2022, 10:38 PMFuETL
11/22/2022, 12:54 PMexcept
statement, that was the cause of this question.Anna Geller
11/22/2022, 1:11 PMimport pendulum
from prefect.engine.signals import RETRY
import awswrangler as wr
def check_if_file_arrived_in_s3():
return wr.s3.does_object_exist("<s3://bucket/example_file.csv>")
@task
def s3_sensor(**kwargs):
bool_s3_object_arrived = check_if_file_arrived_in_s3()
if bool_s3_object_arrived is False:
raise RETRY(
"File not available yet, retrying in 20 seconds.",
start_time=pendulum.now().add(seconds=20),
)