Can I have multiple overlapping instances of a given flow run (let’s saw with different parameters)? If so can I have shared state between those runs (preferable in memory rather than on disk)?

Hi @Joseph multiple flow runs do not have the option to share state. Is there a specific use case you have in mind?

Thanks for the reply. I have a flow which creates jobs that populate partitions of a database. Every time that the flow is run, it may discover new partitions. It is possible that one instance of the Flow will still be running (populating one partition, which can take hours) while the next instance of the Flow begins running. I need some state to track which partitions are: not yet populated, still being populated and have had a failure during population.

While they can’t share state directly take a look into Prefect targets because they will allow you to write data in one flow and other flows can react based on target data present at that location https://docs.prefect.io/core/concepts/persistence.html#output-caching-based-on-a-file-target https://docs.prefect.io/core/idioms/targets.html e.g. Flow A is running and write some data or lock to a known location and then flow B can check that target to see if it is present at that location. If it is it can react appropriately

Got it - thanks for the links, I appreciate it.