Samuel Kohlleffel
08/03/2021, 2:48 PMflow.serialized_hash() to only register the modified flows. However, flow.serialized_hash() is not returning a consistent hash value for flows that have not been modified. Why would this be the case?
For context, I'm testing by registering the flows locally with flow.register()Kevin Kho
flow.register(…, idempotency_key=flow.serialized_hash()) , or are you just running flow.serialized_hash() multiple times and comparing the values?Samuel Kohlleffel
08/03/2021, 2:56 PMflow.register(…, idempotency_key=flow.serialized_hash()) but that wasn't working for me, so then I tried to just test flow.serialized_hash() by running it and comparing the values on independent registration runs.
I also compared the flow.serialize() values on independent runs and those are different values as well.Kevin Kho
Samuel Kohlleffel
08/03/2021, 3:04 PMSamuel Kohlleffel
08/03/2021, 3:55 PMflow.serialize() is the following:
Test 1: "start_date": {"dt": "2021-08-03T10:47:33.780910"
Test2: "start_date": {"dt": "2021-08-03T10:49:13.281844"
Does the time of registration impact the flow.serialize() return value?Samuel Kohlleffel
08/03/2021, 4:01 PM"clocks": [{"__version__": "0.15.1", "cron": "20 6 * * *", "day_or": true, "end_date": null, "labels": null, "parameter_defaults": {}, "start_date": {"dt": "2021-08-03T10:49:13.281844", "tz": "America/Chicago"}, "type": "CronClock"}],Kevin Kho
Samuel Kohlleffel
08/03/2021, 4:08 PMstart_date = pendulum.now(tz="America/Chicago")
clocks = [CronClock(s, start_date=start_date) for s in cron_strings]
run_schedule = create_cron_schedule("20 6 * * *")
with Flow(
name="netchb-to-snowflake",
schedule=netchb_run_schedule,
) as flow:
flow.chain(task chain...)Samuel Kohlleffel
08/03/2021, 4:10 PMstart_date parameter when building the Cron schedule. The start_date value is then used in the flow.serialize() json object which in turn causes inconsistent hash values because we are using pendulum.now(tz="America/Chicago") as the start_date.Kevin Kho
Kevin Kho
start_date to the first of the year for something in the past so that the serialized_hash will stay the same.Samuel Kohlleffel
08/03/2021, 5:11 PM