https://prefect.io logo
Channels
Powered by
b

Ben Muller

11/17/2022, 12:57 AM
Hey if I have a flow like so:
Copy code
@flow
def my_flow(
    start: str = (datetime.utcnow() - timedelta(days=7)).strftime("%Y-%m-%d"),
    end: str = datetime.utcnow().strftime("%Y-%m-%d"),
):
Will these default parameters be set at deploy time or will they update at run time ?
✅ 1
r

Ryan Peden

11/17/2022, 1:29 AM
Run time, more or less. The functions that generate your default params here will run when your flow's file get gets imported by the Prefect engine, which happens just before your flow runs. A few things happen between importing the file and running the flow, but they shouldn't take long unless one of the flow's parameters is a 
PrefectFuture
the flow needs to wait for. If you wanted to set default values set at deploy time instead, you could do it using 
parameters
in a Python deployment, e.g.:
Copy code
my_deployment = Deployment.build_from_flow(
   # other args,
   parameters={
        "start": (datetime.utcnow() - timedelta(days=7)).strftime("%Y-%m-%d"),
        "end": datetime.utcnow().strftime("%Y-%m-%d")
   }
)
You could also set them in a CLI deployment using the 
--param
or 
--params
flags but you'd need to generate the values using shell commands instead of Python. Run 
prefect deployment build --help
if you'd like to see usage instructions for these flags. 🇦🇺 🇨🇦
b

Ben Muller

11/17/2022, 1:30 AM
Got it, that is what I want. It was different in prefect 1, so just making sure. Can I then assume that the default params that show in the UI when you go to change them are deploy time?
r

Ryan Peden

11/17/2022, 1:37 AM
Good question; that they show in the UI makes me want to double check. Perhaps our deployment code is smarter than I'm giving it credit for and runs the defaults at deploy time, saves them, and re-applies them values at runtime. I'd assumed it would follow the normal Python rules, but come to think of it I'm not sure I've actually deployed a flow with dynamic defaults. If I was wrong, I'll find another way to do what you need.
b

Ben Muller

11/17/2022, 1:42 AM
Ok, let me know. Very interested!
r

Ryan Peden

11/17/2022, 2:03 AM
It looks like it works the way I thought. I see this in the UI:
But when I ran the deployment a few times with a flow that prints the start and end parameters I get this:
Copy code
21:00:13.695 | INFO    | prefect.agent - Submitting flow run '1db73ea5-0884-4b77-8ffb-50a949226cc9'
21:00:13.814 | INFO    | prefect.infrastructure.process - Opening process 'eccentric-jackal'...
21:00:13.826 | INFO    | prefect.agent - Completed submission of flow run '1db73ea5-0884-4b77-8ffb-50a949226cc9'
21:00:16.215 | INFO    | Flow run 'eccentric-jackal' - Finished in state Completed()
start is 2022-11-10-02:00:15
end is 2022-11-17-02:00:15
21:00:16.448 | INFO    | prefect.infrastructure.process - Process 'eccentric-jackal' exited cleanly.
21:00:49.034 | INFO    | prefect.agent - Submitting flow run 'fed027d3-4daf-4174-89e8-7d75689de37d'
21:00:49.118 | INFO    | prefect.infrastructure.process - Opening process 'striped-waxbill'...
21:00:49.124 | INFO    | prefect.agent - Completed submission of flow run 'fed027d3-4daf-4174-89e8-7d75689de37d'
21:00:51.787 | INFO    | Flow run 'striped-waxbill' - Finished in state Completed()
start is 2022-11-10-02:00:51
end is 2022-11-17-02:00:51
21:00:52.060 | INFO    | prefect.infrastructure.process - Process 'striped-waxbill' exited cleanly.
21:01:04.212 | INFO    | prefect.agent - Submitting flow run 'd671317f-f0f0-4833-8075-caceee7c9d15'
21:01:04.315 | INFO    | prefect.infrastructure.process - Opening process 'cherubic-skunk'...
21:01:04.324 | INFO    | prefect.agent - Completed submission of flow run 'd671317f-f0f0-4833-8075-caceee7c9d15'
21:01:06.813 | INFO    | Flow run 'cherubic-skunk' - Finished in state Completed()
start is 2022-11-10-02:01:06
end is 2022-11-17-02:01:06
which looks like it is what you want. Or if I misunderstood and you want them set at deploy time, the code I showed for that should work too; those values would show up in the Overrides column in the UI.
We may need to think of a clever way to detect and handle dynamic parameter defaults like this to ensure the UI isn't misleading, so I'm happy you asked about it 😄
b

Ben Muller

11/17/2022, 2:29 AM
🙏 🙏 🙏
z

Zanie

11/17/2022, 2:59 AM
Oof this one is going to be challenging to detect.
b

Ben Muller

11/17/2022, 3:26 AM
Btw @Ryan Peden I want them at run time. Not deploy time. So this is good 👍
So just to make sure I'm following. Defaults are python and dynamic at run time and UI shows deploy time variables (whatever they happen to be static at deploy time)?
z

Zanie

11/17/2022, 6:15 AM
We store the schema of your function when you deploy it. That includes defaults for parameters. At inspection time, dynamic parameters are indistinguishable from their static value at that moment. Flows are deployed as scripts: so to load a deployed function, we execute the code and the dynamic values change. We could easily pass the defaults explicitly from the schema to use the deploy time value. Fixing the display in the UI to show that it's a dynamic value will be far more difficult. Probably makes sense to discourage the pattern entirely unless you are aware of the caveats.
b

Ben Muller

11/17/2022, 6:16 AM
Haha I'm more confused now. Is my assumption correct?
I can test it but you seem to be on top of it!
Honestly I don't mind about the UI. I just hope the default in the python flow is dynamic.
That suits my use case anyway
r

Ryan Peden

11/17/2022, 2:06 PM
Hi Ben, your assumption is correct. 🙂 Those defaults will be dynamic when the flow runs.
3 Views
Powered by