Hi everyone.
We are a small data company based in Europe. We use prefect since months without going on cloud. We built all our automation flows on prefect and it's an amazing tool, thanks a lot for your work.
We decided to move some flows on the prefect cloud backend and we were surprised to see the total duration time increase by a factor of two compared to our local prefect backend server. The flow on which we ran the tests is one of the longest in our stack (191 tasks, 26 parameters).
Keeping the same agent, executor, computer, and parameters. We move from 65s to 135s. Each task takes on average +100% time increase (even the parameters collection).
So, we understand that there is some time cost linked to the exchange of state information and metadata but could you, dear fellow community, please help us with the following questions?
1. Is the time gap between runs on the prefect cloud backend and the prefect server backend proportional: The longer the flow, the longer the time gap?
2. Are there any good practices to reduce this time gap?
3. Upgrading to the standard or enterprise offer will that make things faster?
Thanks a lot :)
Extra information:
Our internet connection at the office 127 mbps/s download, 7.8 mbps/s upload.