https://prefect.io logo
a

Andrea Nerla

01/28/2022, 3:19 PM
Hi folks, I have a problem of failed code passed as success. For instance in the T part of my ETL( which is simply a os.system...) I've put a test and even if it fails Prefect still declares it a success.
(I already know that this is probably a stupid question and I apologize for that)
a

Anna Geller

01/28/2022, 3:31 PM
There are no stupid questions! Can you by any chance share the example flow you were using? Could it be that you have some try/except block that traps the exception without raising it?
a

Andrea Nerla

01/28/2022, 3:32 PM
Copy code
@task  
def extract_load():
    os.system(r'"python C:\Users\andrea.nerla\Desktop\sidal\python_per_test\test_systemuser.py"')
The part that bothers me is extract_load
Sorry I might have understood the problem
a

Amanda Wee

01/28/2022, 3:40 PM
I think the reason is that
os.system
runs the Python interpreter to run the specified Python script, and the exception is raised in that script, which doesn't then propagate to this Prefect task since it is an external script. You might need to say, check for a non-zero return value for
os.system
then raise an exception within the Prefect task, e.g., Prefect's
FAIL
signal.
upvote 2
2
a

Anna Geller

01/28/2022, 3:40 PM
You’re right that this is the part that causes troubles. You can call such tasks using ShellTask - here is an example
a

Andrea Nerla

01/28/2022, 3:52 PM
Thanks a lot girls!
👍 2
5 Views