Hi there
Do you know a method to manually skip a task while the flow is already running? We are using prefect 1.1.0.
We have the problem that sometimes we have very long running tasks (database performance issues). There we would like to skip this running task run and continue with the next task. So that everything else is done. We already set for this clean up tasks in the end
trigger=always_run, skip_on_upstream_skip=False
, but currently we have to abort the full flow run. So this task will never be executed.
@Anna Geller Thanks for your response.
As far as I saw is your article more about handling states in Orion.
In my case, I can not know if the task run will be hanging (it's up to an external system -> DB). So I can not really easy handle, prefects task executer needs to handle this via timeout for example. Or best would be, if we could cancel a running task run manually in the GUI.
And this in Prefect 1.1 not Orion.
I saw that in the Prefect 1 GUI we can overwrite the state of a task run (e.g. to cancelled), but this leads to not ending run.
a
Anna Geller
11/01/2022, 1:21 PM
cancellations are hard in a hybrid model, but your use case would still work with if/else + submitting work to task runner, waiting and operating on state, or on timeout if needed
task timeouts are coming for Prefect 2 (branch: add-task-timeout)
Anna Geller
11/01/2022, 1:22 PM
in v1 you could explore offloading that to a separate flow and using create_flow_run + wait_for_flow_run
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.