I'm trying to decide on best practices for a complex flow schedule. I have a flow that needs to run at specific times each day, but those times change each day depending on data obtained from an API in the morning. Before I moved this flow to Prefect and did it the old fashion way, I just ran the script every minute, and it started with a function call that returned a bool depending on if it should continue or not. (That function would do the complex logic of hitting the API or retrieved data and determine if it should run at that moment). In Prefect, should I create a task that does this check and then skips the rest of the flow if it is not supposed to run? Or is there a way I can create a dynamic schedule that I pass into the flow itself? Or is there a better way altogether?

@Matt Harvey that’s an excellent question. In Prefect, we think of schedules as a stand-in for manually starting your flow — in other words, just a convenience for using the API to create flow runs with known future start times. In Prefect Cloud, I think we’d actually suggest not using a schedule at all for this Flow (because the future start times aren’t well known). Instead, we’d recommend creating a small flow that hits that API on a regular basis and then creates new runs of your main flow with appropriate

start_time

settings. For example, you might hit the API at 9am every morning and create 5 new runs of the main flow in response.

@Jeremiah Thanks for the response. Creating new runs with the appropriate start times feels like a clean solution for my use case because although start times are not known for a given day, I do know when the list of start times becomes available each morning. Would you be able to point me towards the pertinent documentation for creating flow runs on the fly like that?

@Matt Harvey our documentation on the Cloud API is admittedly lacking at the moment and will get an upgrade soon. However, the GraphQL call is documented here (https://docs.prefect.io/cloud/concepts/flow_runs.html#scheduling-a-flow-run) and the equivalent call from the Python client is documented here (https://docs.prefect.io/api/unreleased/client/client.html#prefect-client-client-client-create-flow-run)

Thanks as always. Prefect is a treat to work with.