https://prefect.io
Join Slack
Hello! We want to have possibility to continue the flow run in case K8S pod was killed and flows run...
# ask-community
v
Hello! We want to have possibility to continue the flow run in case K8S pod was killed and flows run restarted on the new pod. Is it possible to use some Prefect features to share that state and results from executed tasks to continue the flow run from where it was interrupted?
✅ 1
c
You can persist results to remote storage, or an underlying storage mount
It would be the same concept as if you did this locally in your environment, with the caveat that pods are ephemeral, so you would need to configure the result persistence manually
v
Could you please explain how it works in more details, because I can't find details in documentation? I see a lot of arguments that I can pass in 
@task
decorator, but I don't understand how restarted flow will know what values should be loaded as result for already executed tasks in the previous run.
c
Copy code
If there is no context to load the storage from and results must be persisted, results will be stored in the path specified by the PREFECT_LOCAL_STORAGE_PATH setting (defaults to ~/.prefect/storage).
since pods are transient, 
/.prefect/storage
will not persist
so you need to persist your storage either remotely, or using a volumeMount in kubernetes for provisioned storage
🙌 1
v
Thanks. Could you please explain how second attempt will know how to find needed results? Will it use flow run uuid + task id?
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by