https://prefect.io logo
Channels
Powered by
q

Q

07/12/2023, 3:52 PM
Yo so I'm trying to follow this doc to configure result storage. My goal is to have two deployments of the same flow use different result storages. But it seems like the changes to 
result_storage
I make via 
.with_options
before calling 
Deployment.build_from_flow
are ignored, i.e. whatever's set in the 
flow
decorator call is always used. Looking at the 
Deployment.build_from_flow
I get the feeling that all the flow parameters (e.g. 
result_storage
) are not saved to db, only the 
entrypoint
is. And the 
entrypoint
doesn't get changed by 
with_options
since it's determined by the name of the underlying (non-Prefect) function, which stays the same. And so this entrypoint gets called when deployment is run, using whatever's set in the 
flow
decorator. Am I missing something here?
d

Deceivious

07/12/2023, 4:05 PM
The in memory python flow that was overriden with with_options is not the one being deployed but rather the flow on the file itself. OR something along that line.
Copy code
@flow
def flow1():
    urstuffhere()

flow2=flow1.with_options(.....)
I believe if you try to deploy the 
flow2
it should work
it should be in the same file
q

Q

07/12/2023, 4:39 PM
Yep, this seems to be what's going on, thanks! The workaround in that thread relies on the variable name staying the same (because 
fn.__name__
and, therefore, the entrypoint don't change), so you'd have to do something like
Copy code
@flow
def flow1():
    urstuffhere()

flow2=flow1.with_options(.....)
flow2.fn.__name__ = "flow2"
But that doesn't work 'cause 
fn
points to the same object for 
flow1
and 
flow2
, so you'd try 
import copy; ...; flow2.fn = copy.deepcopy(flow1.fn)
but 
copy.deepcopy
doesn't copy functions so you'd have to do this. Ugh.
For the same reason if you're creating a flow via 
Flow
class constructor the variable name has to match, so you can't do
Copy code
def func():
    pass

flow1 = Flow(func, **config1)
flow2 = Flow(func, **config2)
d

Deceivious

07/12/2023, 4:53 PM
I have a better solution for u
I worked on a script that copies a deployment with new param
Wait storage is a parameter in deployment.
U don't need to use override?
q

Q

07/12/2023, 4:55 PM
result_storage is not the same thing as storage though
d

Deceivious

07/12/2023, 4:55 PM
Oh right forgot
You can try subflow I guess
Copy code
@flow
def sub():
	urstuffhere()


@flow
def main(storage_id):
	sub.with_options(storage=use_storage(storage_id))()

#deploy main twice with 2 different parameters?
q

Q

07/12/2023, 5:43 PM
That works (at least with a simple flow), although spawning subflows like that seems meh. This one works as well, gonna have to see. Appreciate the help 👀
Powered by