https://prefect.io logo
Title
a

Alvaro Durán Tovar

05/20/2022, 8:42 PM
if I want to create flows dynamically like this (registration happens after)
def build(...):
    with Flow(...) as Flow:
        ...
    return flow
is it possible to use docker storage? thinking on possible issues trying to find the path of the flow inside the docker file, there won't any "flow" variable on the module level
k

Kevin Kho

05/20/2022, 8:45 PM
Yeah if you used script based storage, the flow object needs to be present in the global namespace. I think there are some scenarios where this works and some where it doesn’t. Are you building the Docker container ahead of time or Prefect does it for you?
Are you using script based storage? Or pickle based?
a

Alvaro Durán Tovar

05/20/2022, 8:54 PM
script
im building it before registering the flows
I mean for this case is fine to pickle I guess if that will work
but mmmm yeah i see that must happen during building the docker image
mmm what about gcs + pickle? is that possible?
k

Kevin Kho

05/20/2022, 8:57 PM
Unsure why you can’t just have:
def build(..):
    ..
    return flow

flow = build()
and it will be in the namespace? It doesn’t do anything anyway
Ah actually never mind. You lose dynamicism. I would do pickle based storage and yes you can on GCS and on Docker
a

Alvaro Durán Tovar

05/20/2022, 9:46 PM
Thank you!
a

Aaron Goebel

06/01/2022, 6:17 PM
@Alvaro Durán Tovar are you dynamically building this flow in response to something like an API request?
a

Alvaro Durán Tovar

06/01/2022, 6:18 PM
No, I'm doing this on CI/CD pipeline, but I don't think doing on a http request will make a difference