Hi, I have a few questions about managing different versions of flows: • Is it possible to get the flow's numerical version number and project name during a flow run, either from the

prefect.context

object or elsewhere? • The flow ID used in the URL https://cloud.prefect.io/paravision/flow/<ID> seems to be different from the flow ID that's output when registering a flow. Is it possible to get that ID during a flow run through

prefect.context

?

prefect.context.flow_id

looks like the same one that's returned when registering the flow, but would like to also get this other flow ID. • When registering a flow, it increments the version number and archives the previous flow version. Is it possible to un-archive a previous version so it can be run again?

Hi @Aric Huang, The stuff in the context can be found here so you can find version under `prefect.context.get(“flow_run_version”). There is a flow group id, a flow id, and a flow run id. The flow group holds all versions of the flow. This doesn’t look available in the context so you need to the use the GraphQL API to get that. Not quite unless the unarchived version’s code can be accessed. If the intent is to alternate between versions, I think parameters might be able to do that for you in some way.

@Kevin Kho Thanks, I am seeing some odd behavior with

flow_run_version

however. The number returned doesn't match the flow's version number in the UI, for example:

Oh you’re right. Looks like it can’t be unarchived unfortunately. Just to let you know, user versions will be supplied in the future as opposed to a number that increments. This current versioning is not that helpful in a lot of cases. I was hoping you’d be able to rollback by removing later versions, but I don’t even think you can like that unfortinately.

I see, do you happen to know if unarchiving/rollback is something that can be supported in the future or if there's a design decision against allowing that?

We are working on it, but the shape is not that definite yet from what I know. This is a common painpoint though that we’ve heard a lot and we’re aware that the versioning is not that helpful as is

Got it. By "user versions" do you mean we'll be able to specify a version number/string when registering the flow in the future? If so and Prefect supported a way to run a specific flow version that would be very useful for us, since we are running flows that produce benchmark results and may want to re-run an old version or refer back to the previous code later on

Yes that’s exactly right. You provide the version.

Great, thanks for the context here

But in the mean time, I don’t know if an artifact registry like MLFlow would help you to keep track of this kind of experimentation

Yeah that's a good thought, we are looking into different ways to track experiment results and something like MLFlow looks like it could be helpful

I’m looking into https://wandb.ai/site for tracking experiment results

What is your impression so far?

Getting demo next week, haven’t had time to start trying to use it yet (we are weeks away from delivering our model for this year - kinda distracted 😉 )

@Kevin Kho we were also looking to retrieve the same information out of the prefect context, especially flow_version. Other field that we are interested in are - • Team name (we have 3 team -dev/test/prd) • project name

i think you may need to use the graphQL API for that because those are not necessarily things related to the flow run context

Thanks @Kevin Kho. Trying out graphQL - would you have queries for extracting team/tenant name & project id for a flow run?

Try this:

query {
  flow_run (where: {id: {_eq: "6082914d-c89a-40ca-af1c-8c77e9827475"}}){
    id
    flow {
      project {
        name
      }
      name
    }
    tenant {
      name
    }
    }
  }

Thank heaps @Kevin Kho. That worked.

The context gets copied a couple of times throughout the Flow lifecycle so I don’t think we just want to necessarily add everything to the payload because it would bloat stuff also

make sense. Thanks for the quick response @Kevin Kho

No problem!

@Kevin Kho we are trying to get the flow info(tenant & project name) to add it to logs within a flow. We can get the information using GraphQL api & python requests. Wondering if there is a way to skip authentication to the API within a flow run. or way to pass in authentication header using secrets within flow. Flow must be using SA key to communicate with the key

You should be able to use

from prefect.client.client import Client

client = Client()
client.graphql(query)

right?

Cool - will give this a go. Thought we need to use API key for cloud, as mentioned in the doco -https://docs.prefect.io/orchestration/concepts/api.html#authenticating-the-client-with-cloud

I think it will pull it automatically if you don’t supply the key. You flow must be authenticated somehow.

Thanks @Kevin Kho that worked. We have all the necessary information related to the flow-run getting logged now.