Hello! - Task question: is it possible to 1. re-run a task w/o triggering downstream dependencies? 2. do the same with ad-hoc task parameters?

Hi @Simon Basin! There isn’t a first class way to do this, though running partial flows is on our roadmap (your feedback on this issue on the topic would be greatly appreciated!: https://github.com/PrefectHQ/prefect/issues/2269). To do this now, you would need to surgically set the downstream tasks of a flow run to be in a “Finished” state to prevent their triggers from being checked (either directly, or with some manipulative caching configuration on the flow). For simple or testing purposes, you can run individual tasks outside of a flow context (see https://docs.prefect.io/core/advanced_tutorials/task-guide.html#running-tasks), but I think you want to be re-running them in a way that affects a flow run if I understand you correctly.

thanks for the info! This functionality would be "a must" for us. Some typical scenario: an ETL flow task is failing and we need to re-run it with custom arguments triggering (or not) dependencies.

Gotcha! If you can any share more, I’m wondering if avoiding the downstream dependencies in your use case is about having a manual or human step to check if you want to proceed specifically because the parameters may have changed (we are usually working from an assumption that when upstream tasks succeed downstream tasks want to be run). I’m also curious if this failing step in the ETL is deterministic or could be any task in the flow, and if it is safe to assume whatever task it is the failure is intermittent.

let's split the two use-cases here: 1. run failing task with ad-hoc argument(s) 2. run task w/o triggering dependencies (both can be done in Airflow or Teamcity, or ActiveBatch)

I’ve definitely done (2), run individual tasks in a dagrun context in Airflow with

airflow run

, but how do you do number (1) with it? I haven’t used Teamcity or ActiveBatch, I will add them to our running issue as potential research topics for inspiration! Thanks!