Hi Everyone, I'm having an issue with registering flows. Whenever I register the same flow, it bumps the version, even if the metadata is unchanged. I've tried using the CLI as well as flow.register(). I've also tried using flow.register('project_1', idempotency_key=flow.serialized_hash()) and it still bumps the version. I am using a local agent, local storage and the cloud backend. Is there another configuration that needs to be set so this doesn't happen? Or should this all be done in docker?

Hey @DK, I don’t think moving to Docker will do this. Will ask the team to be sure but what is your use case that you need the version pinned? We might be able to solve this some other way.

I once tried with cli command

prefect register flow

it doesn't bump version, but with

prefect register

it does

Actually that’s right @Ranu Goldan! I know some people on the other side have been asking why their flow is not bumping version 😅

@Kevin Kho when should the flow version be bumped? In my ECS-based setup with a bunch of flows packaged together with a local agent, I build the Docker image each time we make a change, even if it is only to tweak a single task. When the Docker container starts up, it runs a shell script that runs the Python scripts that define and register the flows (i.e., not using the CLI). Like what @Ranu Goldan observed, with

serialized_hash()

the flow versions kept increasing even if it was just because the ECS task was restarted, thereby starting up the Docker container again. My suspicion was that some Docker container metadata was changing each time, e.g., something host-related, so the serialized hash computation changed. My solution was to write my own hash function that took into account the dependencies (including Prefect version) and the code content, but I wonder if that is overkill.

Hey @Amanda Wee, correct me if I’m wrong but I think what you’re referring to is that DockerStorage was not respecting the cache and rebuilding everything. Have you seen this recent PR that fixed it?.

No, I'm using S3 storage. The Docker image is for ECS, not for Prefect.

Oh I see what you mean. Ok I’ll ask the team and get back to you

@Kevin Kho It was my understanding that the version should only be bumped when the metadata changes. My use case is to automate the registering of flows once they are added or changed by a developer. I wrote a function that looks in a directory and collects all flows and registers them. I would like to not have the versions bumped on flows that have already been registered and have had no changes. Interestingly enough, I tried testing again this morning and discovered that in using the CLI or python api to register, the version will get bumped every other time. For example, if I run

prefect register --project Project1 -p src/register.py -n Flow1

it will register the flow, then if I run the same command a few seconds later, it will skip with the message: "Skipped (metadata unchanged)". However, running it a third time, it will register the flow as version 2. So it seems to be skipping the registration as desired every other time.

Hi @DK - this sounds like an issue with CLI registration; I'm going to open an issue from this thread for the Core team to look into.

Thanks @nicholas. Just to clarify, I'm not sure if it's limited to the CLI because I was getting the same/similar issue with the python api and using the idempotency_key=flow.serialized_hash() parameter. That being said, I think was able to resolve the issue for now. I'm not exactly sure how, but I reorganized my code and took out some control flow (case, merge), and registering seems to be working as desired and not bumping the version. However, I'm not completely sure if that was the culprit because I had a test Flow with a couple basic tasks without control flow and it was doing the same thing and I resolved it by removing the logger from the task. Though I added it back in and it still worked correctly. I'm assuming there was likely some user error on my part. If I come across it again I'll comment on the issue.

Thanks @DK !