I found this library particularly helpful for deferring flow configuration until it is actually needed. This lets me import the
flow
object without having
MY_BUCKET
secret available (e.g. for generating documentation), while still materializing the config correctly when the flow is run or registered. Curious if anyone has seen similar use cases or solutions?
Copy code
from prefect.client import Secret
from prefect.storage import S3
from lazy_object_proxy import Proxy
flow = Flow(
...,
storage=Proxy(lambda: S3(Secret('MY_BUCKET').get())),
)
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.