I have a pipeline with many steps to be implemented in Prefect. Is it possible to start the pipeline from a specific step in the middle if necessary? For example, if there's an error in a step, I'd like to fix the step and start from there instead of running from the start.

Hi @Varuna Bamunusinghe, the answer here is not quite. First, in order for changes to take effect, it means that you need to re-register the Flow. If you re-register and the version goes up, you need to start a new Flow run. So what will help you is caching. targets are a file-based persistence so if the next flow run happens and it sees that the file exists, it won’t run that task anymore

Thanks @Kevin Kho. This should help.

You can also try breaking this up into a Flow of Flows and then use the

StartFlowRun

so you can trigger the subflow