https://prefect.io
Join Slack
if I want to create flows dynamically like this (registration happens after) ```def build(...): ...
# ask-community
a
if I want to create flows dynamically like this (registration happens after)
Copy code
def build(...):
    with Flow(...) as Flow:
        ...
    return flow
is it possible to use docker storage? thinking on possible issues trying to find the path of the flow inside the docker file, there won't any "flow" variable on the module level
k
Yeah if you used script based storage, the flow object needs to be present in the global namespace. I think there are some scenarios where this works and some where it doesn’t. Are you building the Docker container ahead of time or Prefect does it for you?
Are you using script based storage? Or pickle based?
a
script
im building it before registering the flows
I mean for this case is fine to pickle I guess if that will work
but mmmm yeah i see that must happen during building the docker image
mmm what about gcs + pickle? is that possible?
k
Unsure why you can’t just have:
Copy code
def build(..):
    ..
    return flow

flow = build()
and it will be in the namespace? It doesn’t do anything anyway
Ah actually never mind. You lose dynamicism. I would do pickle based storage and yes you can on GCS and on Docker
a
Thank you!
a
@Alvaro Durán Tovar are you dynamically building this flow in response to something like an API request?
a
No, I'm doing this on CI/CD pipeline, but I don't think doing on a http request will make a difference
6 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by