Hi! I'm interested in measuring percentage of task...
# prefect-community
Hi! I'm interested in measuring percentage of tasks in the pipeline that were read from the cache. The only way to obtain that information that I can think of would be to have a shared counter that would be incremented at the beginning of each task. I believe that global variable won't work when running in distributed mode so I was wondering about an alternative. Is it possible to modify prefect.context from within a task? And then access such accumulator in a task that runs at the very end and prints the result/saves it somewhere?
You’ll probably want to query the graphql api instead
this should get you the cached tasks for instance
Copy code
query {
      where: {
          state: {_eq: "Cached"},
          flow_run_id: {_eq: "<flow_id>"},
    ) {
and then to get the denominator - remove the
clause/filter (assuming you want to divide by all your tasks )
Hey, thanks @Marwan Sarieddine for suggestion! But from what I've seen (and documentation seems to support that Cached state indicates that tasks output has been cached - not that the task has been read from cache (although in that case the state is going to be Cached too) https://docs.prefect.io/api/latest/engine/state.html#cached
I see - yes you are right @Krzysztof Nawara