https://prefect.io logo
Title
k

Karlo

11/10/2022, 10:40 PM
Hey guys, I’m new to prefect development. Our team is still on v 1.3 How can I fail a task if the task log contains an error? Had a couple flows marked “Success”, but they actually failed in a certain task, the actual errors are recorded in the task’s log and are ERROR level but don’t seem to fail the task.
:upvote: 1
m

merlin

11/11/2022, 6:10 AM
I've been working with tasks/flows that need to be more hands on here. My starting point was this: https://docs.prefect.io/concepts/flows/#return-a-manual-state Basically a flow completes if there is no exception raised. You can return a manual state of Failed() after applying some logic, for example after testing a variable. In my example, I'm testing a flaky service connection before running a subflow. By returning cancelled to the main flow the flow run shows as Cancelled in the UI. Not the rules on multi-state flow runs, they will get flagged as Failed if one is Cancelled and the others Completed. There is an issue open about this behavior.
# VPN_connection is another task that tests a DB service connection

@flow(name="bulk-extracts")
def bulk_extracts(directory_name, scan_minutes):
# TODO: use map to enable concurrent flows
    logger = get_run_logger()
    extract_folder_items=Path(directory_name).iterdir()

    <http://logger.info|logger.info>("Conditional flow dependent on VPN connection begins.")
    if VPN_connected(time_period_to_scan_minutes=scan_minutes):
        for item in extract_folder_items:
            <http://logger.info|logger.info>(f"daily extract file: {item}")
            extract_config = ExtractConfig(filepath=item)
            with tags(extract_config.dataset_name, "extract"):
                extract_write(extract_config)
    else:
        return Cancelled(message="VPN connection failed, cancelling flow")
:gratitude-thank-you: 1
c

Chris Reuter

11/11/2022, 12:31 PM
Looks like Karlo is on v1.3 and subflows are v2 - maybe @Kalise Richmond can help answer this one
k

Karlo

11/11/2022, 11:09 PM
Bueller? 😉
k

Kalise Richmond

11/12/2022, 1:00 AM
Hi @Karlo on 1.0 you should be able to control the task state at a granular level with Signals and set it to FAIL if there is an error in https://docs-v1.prefect.io/core/concepts/execution.html#state-signals
🙌 1
:thank-you: 1
k

Karlo

11/12/2022, 4:48 PM
@merlin Thank you I will keep this in mind when we upgrade to 2.0