https://prefect.io logo
m

Matthew Seligson

01/26/2022, 7:03 PM
What means are available for tracking task runs across different versions of a flow? If I have a flow with one task, and I reregister the flow, the task’s ID will change.
k

Kevin Kho

01/26/2022, 7:05 PM
Maybe the task name would help?
m

Matthew Seligson

01/26/2022, 7:12 PM
I’m wondering what facilities are built into Prefect to support this. For example I’d like to be able to change the task name. If I do that, the task id will change. What do I have available to determine the previous ID?
k

Kevin Kho

01/26/2022, 7:17 PM
We don’t know the task content though so I don’t think we can support that other than the
name
or
slug
. We also have
tags
if that helps.
Could you tell me more about the use case? Are you trying to do something like load all task runs across the Flow run history?
m

Matthew Seligson

01/26/2022, 7:32 PM
Thanks Kevin. In a simple case I have a flow with one task that checks network activity and either fails or succeeds, and I run the flow every day for a year. The next year I add a new task to the flow to send an email indicating the result (success or failure) of the first task. My new flow has two tasks, the network task and the email task. However, the task run history for the network task is empty because it is technically a different task with a different task ID. I want to see the full run history of the network task.
a

Anna Geller

01/26/2022, 7:52 PM
I second Kevin’s suggestion to use
tags
for such use case - imo it’s probably the cleanest approach and it will work nicely in Orion. But even using the task name or slug can be used in your use case. It’s all a matter of combining data from multiple queries. For example, this is how you can get all the versions of a specific flow:
Copy code
query {
  flow(where: { name: { _ilike: "%dask%" } }) {
    id
    name
    version
    tasks {
      name
      id
      slug
      tags
    }
  }
}
you can replace the “dask” text with some part of your flow name. And then to get the task run history for that, you would need another query. But note that storing logs and the history of your flow and task runs over such a long period of time (you want to analyze a period of potentially two years of run history?) would be difficult or at least expensive. For example, Prefect Cloud standard plan has a 2 weeks of log retention.
in short, even if your task id changes, you can still retrieve this data via name/slug/tags to get the history, as long as your backend still has that history (based on your retention period settings)
3 Views