Samuel Hinton
09/18/2024, 4:40 AMdeployments = []
for our_flow in list_of_our_flows:
flow_from_source = await our_flow.from_source(repo, entrypoint=entrypoint) # Takes 2s to run
deployments.append(await flow_from_source.to_deployment(...)) # Takes 4s to run
await prefect.deploy(*deployments, ...)
With about 100 flows/deployment combinations, that's 10 minutes of registering flows, all belonging to the same repo, and given we have a dev env before prod (ie we deploy twice), our pipeline to update a flow has broken 20minutes.
Does anyone know either why making a deployment takes multiple seconds, or if theres a way of doing this differently or somehow moving the <http://flow_from_source.to|flow_from_source.to>_deployment
into a batch?
I'm also not sure on best practises here, like should I do what I found in one tutorial (to_deployment and then prefect.deploy, or just flow.deploy, or something else). Our use case is we have one repo, which contains 100 or so flows, each that have a RRule pattern on them, and we want to make one deployment per flow+schedule pair. It seems like cloning things 100 times isnt ideal, but it seems to be what the documentation suggests?William Jamir
09/18/2024, 10:07 AMNate
09/18/2024, 2:12 PMSamuel Hinton
09/19/2024, 3:39 AMSamuel Hinton
09/19/2024, 3:57 AM