Hi! I've run into an issue with parameters set in ...
# prefect-ui
y
Hi! I've run into an issue with parameters set in Cloud UI. When I update existing parameters via UI it doesn't take effect immediately. As far as I understand the reason is that Prefect Cloud schedule 10 runs upfront. So right after parameters update I need to wait 10 runs before my changes will be applied. Or I need to change either something in a Flow's code, or in a Flow's schedule so Prefect Cloud will re-schedule this 10 runs with updated parameters. It seems very strange behaviour. Is there any way to force this "re-scheduling" after each update in parameters via UI? To be more precise. Let's say I've got such Flow:
Copy code
import pendulum
from datetime import timedelta
from prefect import Parameter, task, Flow
from prefect.schedules import Schedule
from prefect.schedules.clocks import IntervalClock

schedule = Schedule(clocks=[IntervalClock(start_date=pendulum.datetime(2021, 9, 28, 8, 0, 0, tz="UTC"), interval=timedelta(minutes=10))])

params = {
    "message": "message version 1"
}

with Flow("test_changing_parameters_from_ui", schedule=schedule) as flow:
    p = Parameter("params", default=params)
    task(lambda x: print(x))(p["message"])
I register it in Prefect Cloud. And right after registration Prefect Cloud schedules 10 runs of my Flow. After that i change
message
in this Flow via Cloud UI. For example, set
"message": "message version 2"
. I expect that in the next run updated message will be printed for me. But in fact I need to wait for 10 more runs before i'll be able to see my "message version 2".
k
Yeah you can turn off the schedule then turn it back on. This is because those runs are already created like you mentioned. Turning off the schedule then turning it back on will recreate the runs with the new parameters
y
And is there any option to not create 10 runs? Just start it without any pre-scheduling?
k
Yeah
flow.register(set_schedule_active=False)
y
Ah, thank you, will try
Well it worked for a while but now it is kinf of broken. Some flows are updated with new parameters after turning off/on a schedule. Others aren't. It wasn't comfortable way to deal with parameters updates but now it is just unreliable 😞
k
What is your Prefect version?
Let me give this a test
y
0.15.6
k
Do you really need to set the parameters in the UI? If you do it through code and re-register, the runs will be re-scheduled
I am not seeing how the runs wouldn’t be updated with turning off/on the schedule. Could you give me more detailed instructions to try to replicate?
y
We're setting default parameters in code. And tried to move as much variables to parameters exactly because we wanted to be able to change them through UI - without a necessity to commit changes in repo and run registration process for every small change. Detailed steps: 0. There is an old Flow with many versions and a parameters in JSON. It's scheduled to run daily and I can see that "upcoming runs" are using original parameters. 1. I want to change some field in parameters, so i open this Flow->Settings->Parameters, make changes and press "Set params" button 2. After that I go back to "Overview" and switch off a schedule in UI with a toggle. 3. Waiting for all upcoming runs to dissappear. 4. Turning a schedule back with a toggle 5. Waiting for upcoming runs to appear 6. Opening any upcoming run and checking parameters of this run. I expect to see new version of parameters. But for some flows it's not working and i still see old version. And yes, i wrote a simple Flow to test it. Yo can see it above. And all steps are working fine with it. But not with some other flows 😞 And I think that this issue appeared not so long ago - it worked fine like two weeks ago or so...
k
Ok thanks for that. I will try to replicate, and also check if there were any recent changes that may affect this. If I can I’ll file a bug. Will get back to you either way
I think there are two ways to do this (image below). The first is on the Schedule and the second is on the Parameters. So I guess there are 3 places you can set it with the 1. Flow registration 2. Schedule and Parameters 3. Parameters tab From what I have seen, the order or priority is 1. Get the parameter defined on the schedule 2. Parameter defined on parameter tab 3. Parameter defined on the flow Are you setting it on the schedule level as well? potentially overriding the Parameter value?
y
Hmmm, looks like thats it. I didn't set parameters in schedule explicitly. But as i can see if i create new schedule parameters are copied to this schedule. And overwrites parameters in "Parameters" tab 🤔 It is unexpected, tbh...
And yes, all flows with this issue have "new" schedule
k
I think it can be unexpected in a sense that you don’t immediate know which one takes priority, but after thinking a bit, I think it makes sense to provide places to override the Parameter value (irrespective to schedules) , and also attached to the schedule. You can create multiple schedules with different parameter sets through the UI
y
Yeah, i mean, that it is not obvious that creating a new schedule (even without touching parameters) will make "parameters" tab useless until this schedule exists... I'd really suggest to clarify it in UI or in docs...
k
I see what you are saying. I can open an issue for that in a bit