@Marvin Is there a way to inject Prefect's variable to deployment's scheduled flow's run dynamically and have the flow parameters automatically inferred based on Prefect variable?

To dynamically inject Prefect variables into a deployment's scheduled flow run and have flow parameters inferred, you can utilize Prefect's scheduling and parameter management capabilities. Here's a concise overview: 1. Scheduling: Prefect supports various schedule types such as Cron, Interval, and RRule, which can be defined in a

prefect.yaml

file or via the interactive

prefect deploy

command. This allows you to set up when your flow runs should occur. More details can be found in the Prefect scheduling documentation. 2. Parameter Inference: When deploying flows, you can pass parameters that the flow function will use. Prefect uses Pydantic models to validate parameters, and you can define default values or infer them dynamically through Prefect variables or environment variables. This parameter handling is managed in the flow's function signature, as seen in the source code for Prefect's flow management. 3. Prefect Variables: You can use Prefect's context variables to dynamically adjust parameters at runtime. This is typically done by accessing Prefect's runtime context within your flow or task to fetch and use these variables. For more details on setting up your deployment and managing parameters, you might want to check out the Prefect YAML configuration documentation. If you're using Prefect 3.x, ensure that your flow and task definitions are compatible with the latest API changes. For your specific use case with PyO3, consider how Prefect's Python-based configuration can interface with your Rust integrations. If you have more specific requirements or need further assistance, feel free to ask!

You can use the templating syntax in prefect.yaml, but that would get rendered at deployment time. If you want it to get rendered at runtime what I would do is just pass the name of the variable to the flow as a parameter, and then just load the value inside the flow.

@Nate i see, that's not ideal for my use case. For a scheduled repeated ETL job, we want to be able see that different flows on UI's parameter is adjusted automatically based on variable modified in the previous runs. Not a big deal for now since we can put them to flow's name instead

feel free to open an issue if you think the behavior should be different!

created: https://github.com/PrefectHQ/prefect/issues/17373