Hey all! I have a Prefect flow that runs our Airbyte connections. However, we have hard-coded a dictionary of

airbyte_connection_ids

into the flow. I want to reference an externally stored dictionary (i.e. a

.json

file stored in S3) so that it's easier to change when we redeploy Airbyte connections & get new

airbyte_connection_ids

. Is this possible? Can a Prefect parameter take values from an external source? We are running on prefect version 1.2.4

you can probably build it that way through the deployment by loading in the json file, parsing it (within the deployment.py) then using that as input to create your parameters. You can also use **kwargs as input parameters : https://docs.prefect.io/concepts/deployments/#parameters-in-deployments

@Christopher Boyd makes sense with Prefect 2.x! Any ideas on how to do the .json parsing with prefect 1.2.4? I don't believe deployments aren't a feature of prefect 1.x

Prefect 1 should just allow the raw json

I see.. so I shouldn't run into any issues getting the .json from an external source, either? The static k:v is what I'm trying to get away from.