Hello again,
I am soon to submit a large number of flow runs to prefect cloud (on the free "dev" tier) and wanted to make sure I don't overstep & put too much load on your servers. In summary, how many flow runs would your team need a preemptive heads-up for? Is there a limit to the free tier where I begin abusing it?
In my case, I will be submitting these flow runs without scheduled run dates (so Agents are directed to run them all immediately and in the order they were submitted). This will be on the order of 10,000 flow runs at a time, but in the future this may increase by a factor of 100. I do not need long-term storage of the flow-run metadata either -- so if it is easier/cheaper for your team to delete my flow-run metadata every week or month, I can have a scheduled "clean-up" workflow to do this. Also yes, flow-run concurrency would be nice in this scenario but I am unfortunately not in a position to leave the free tier.
Alongside this, is there a limit to prefect Agents and how often they are disconnecting/reconnecting? I expect to have 20-30 agents that will be terminated and restarted every week. This is because the Agents will be running on third-party HPC clusters within a slurm jobs with timelimits.
-Jack