Hey all, hopefully this is isn't just noise. Have the need to integrate workflow management software into my multi-tenant SAAS software. Trying to figure out if Prefect is an okay match for my use case. I have a preset list of flows/tasks I want each tenant to be able to schedule / run independently. Each of these flows need to be parameterized or use context so I can specify things like "sftp connection info", "db connection info", which will differ per-tenant. Some of these flows must be limited to concurrency=1-per-tenant just to throw a wrench into things.
If anyone has some thoughts / suggestions, would be very happy to hear
01/29/2021, 3:27 AM
Hi @Adam Brusselback - everything you've described is 100% doable on Prefect Server with the exception of concurrency, which isn't available in the open source platform. You could implement some basic concurrency logic in your application though by querying for flow runs in a
state that fit your tenant criteria and prevent ad-hoc runs from being launched.
01/29/2021, 4:18 PM
Thanks @nicholas, after poking around with the docker image / digging through the code, looks quite promising! I know there is likely conflict between enhancing the open core in ways that step on the toes of cloud, just wondering how open Prefect is to contributions around the areas of interest for my company (users, auth/access control, etc)?
01/29/2021, 4:38 PM
Typically our stance has been that adding those to the open source introduces complication that doesn't make sense given our objectives for the tool. Given the fairly diverse auth/access control patterns/needs of organizations, a solution to this in the open source isn't something we're interested in maintaining, since we've created our opinionated version of this with Cloud. That said, we reserve the right to change our stance on that in the future! Hopefully that answers your question but happy to clarify more if needed!
01/29/2021, 5:09 PM
It makes sense for your company which is why I asked. Thanks for the info!