Hi all, new to prefect - quick question. In the past our team used Kylo (a NiFi abstraction) for basic workflows and Airflow for complex workflows. One key requirement is to have a way to define a "template" pipeline. lets say pull from blob storage and dump to staging database. Things like blob storage location and database table names may change and should be tied to a particular run. Kylo did a good job of this; a self service for end users to create flows from NiFi templates - users could create the entire pipeline from UI using the template as base. The Kylo project hasn't been maintend much. Airflow doesn't fit this model as dags have to be developed for each new run instance unless you get into dynamic dags etc. Does Prefect offer something like this in the prefect UI? Let me know if this makes sense or a more detail example is needed.

Depending on how flexible the templates need to be, one pattern we use a lot are flows that define some abstract logic that is configured using something like JSON or SQL or something else depending on our needs. As an example, we have monitors that need to run every day, they query our db, and post messages in various formats to Slack and files in Box. The flow itself doesn’t know anything about the monitors, it pulls a github repo where all our configurations exist and then executes them. The monitor is defined in SQL with some metadata to configure the target Box folder, Slack channel and even message format.

Hmm yeah thats a good point at dev layer. assume JSON - that is good to define the input parameters. like Parameterization, but built into the UI during flow execution. Something that is simple for end user. Else user would have first upload a custom JSON to a database etc - which is totally fine at developer level. What we are seeing more now is users wanting to create flows on their own. Interface / Impl pattern. the data engineer defines the interface of the workflow , and the UI allows users to create it, pass in their params, schedule it, and monitor. Does this exist or is there plans for it?