Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

hey everyone - separate question: I have a workflow and in a very rare instance need to run 3000 runs in parallel. Is there an easy way to point the worflows at a different env at at the time of execution

I think Parameters would solve your use case but I can’t say for certain without more context :thinking_face:

so I have created a financial analysis workflow that pulls data from an API then performs a series of calculations. However the process can only calculate returns for a single day and I need to generate returns for multiple days. My first though is if I have a server/dask cluster big enough, I can run X number of workflows in parallel

this is to prevent building a separate workflow

Ah I see! You could parameterize these workflows to take in a date/date range for which data to process. And you could have a default time range set if you are running on a schedule

And yeah having a large dask cluster to parallelize your runs would allow you to execute many simultaneously