Has there been any discussion of writing Prefect blocks into configuration files? We can reference them in
prefect.yaml
, but actually managing them occurs solely within the UI or scripts.
n
Nate
10/11/2023, 5:01 PM
hi @Dominic Tarro - what sort of use case do you have in mind? by design, blocks are written to the database, but they are effectively just fancy pydantic models, so one could easily write them to whatever format would be helpful if they had a reason to
d
Dominic Tarro
10/11/2023, 5:31 PM
We've been consolidating and standardizing much of our infrastructure, which conceptually we lump blocks into. Right now we have a bunch of scripts with the block definitions in them, but 99% of those only use string literals and environmental variables. Some blocks load an external JSON file. It seems natural for blocks to be persisted in a non-python configuration file like
prefect.yaml
or a separate file that can be passed as an argument to the prefect CLI.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.