Hi All, We're considering moving our ETL processes...
# ask-community
f
Hi All, We're considering moving our ETL processes from Prefect server to the hybrid model for a number of reasons. Currently, Prefect server is running in a Docker container on Azure, and we would like to move our current Flows to a Kubernetes cluster on Azure as well. Is there a definitive way this can be accomplished safely? Also, are there best practices documentation on handling failures or querying states from GraphQL? Thanks very much.
k
On the migration, it should just be a matter of changing the endpoint of your API to cloud, and then everything should be able to run (after re-registration) as normal What do you have in mind for handling failures or querying states? There is a couple of ways around that.
🙌 1
f
Right now I'm using a webhook to notify a Flow failed using the
on_failure
parameter in Flow, but we need something more robust that would return more information like when the flow last failed or succeeded.
k
I would use a State Handler combined with the KV Store to persist the values of last execution time and update it
f
Okay!! So what can I do with GraphQL that can't be accomplished with State handlers?
k
I would say GraphQL is after the flow run while state handlers are during the Flow run, but there are use cases to use GraphQL inside the state handler as well.
You can do the above “last executed” operation by querying the API instead of storing it with the KV Store
f
Fascinating! I will try to find documentation on it as well, you've been a great help. Thanks, Kevin.
k
Of course!