Hi - What is the suggested `target` pattern for a ...
# prefect-community
i
Hi - What is the suggested
target
pattern for a
task
that is called twice in the same flow but without mapping? I currently am mucking around with this pattern below and thought of using tags to differentiate them.
Copy code
duplicate_task_target = "{parameters[<A FLOW_PARAM>]}/{task_name}-{???}"
But that would mean keeping track of duplicate tasks ... which is burdensome when working on multiple flows that then get updated. Any other thoughts? Aside: I think this goes to param based targeting mentioned in other threads.
c
you could use a timestamp; you can use
{date:%strftime-%formatter}
with your favorite datetime formatter
also friendly reminder that the ability to cache results based on inputs has been present for a while with the still-supported
cache_validator
/
cache_for
interface
k
Full list of options in addition to the timestamp mentioned by Chris can be found in Prefect Context: https://docs.prefect.io/api/latest/utilities/context.html
upvote 1
i
(1) Timestamp doesn't differentiate the param inputs. (2) Aren't all the
cache
arguments getting deprecated? https://docs.prefect.io/api/latest/core/task.html#task-2
c
We are considering moving those arguments to the
Result
interface and off the task interface: https://github.com/PrefectHQ/prefect/issues/2619
i
Ok. Cool. So the functionality wont go away
c
yea, correct
i
@Chris White So now that I tried
task_run_id
🙂 and
task_tags
. I am Still puzzled how to differentiate task runs and cache their unique results. With
Copy code
cache_validator=partial_inputs_only(validate_on=['x', 'y']))
then previous result will be overwritten?
c
Hey itay sorry just seeing this; the
cache_validator
interface is currently completely independent from the
target
interface, so it doesn’t write any data to disk. We’re hoping to unify the two in a more intuitive way but in the meantime I think using tags is the best way to go (for now).