i

    itay livni

    2 years ago
    Hi - What is the suggested
    target
    pattern for a
    task
    that is called twice in the same flow but without mapping? I currently am mucking around with this pattern below and thought of using tags to differentiate them.
    duplicate_task_target = "{parameters[<A FLOW_PARAM>]}/{task_name}-{???}"
    But that would mean keeping track of duplicate tasks ... which is burdensome when working on multiple flows that then get updated. Any other thoughts? Aside: I think this goes to param based targeting mentioned in other threads.
    Chris White

    Chris White

    2 years ago
    you could use a timestamp; you can use
    {date:%strftime-%formatter}
    with your favorite datetime formatter
    also friendly reminder that the ability to cache results based on inputs has been present for a while with the still-supported
    cache_validator
    /
    cache_for
    interface
    Kyle Moon-Wright

    Kyle Moon-Wright

    2 years ago
    Full list of options in addition to the timestamp mentioned by Chris can be found in Prefect Context: https://docs.prefect.io/api/latest/utilities/context.html
    i

    itay livni

    2 years ago
    (1) Timestamp doesn't differentiate the param inputs. (2) Aren't all the
    cache
    arguments getting deprecated? https://docs.prefect.io/api/latest/core/task.html#task-2
    Chris White

    Chris White

    2 years ago
    We are considering moving those arguments to the
    Result
    interface and off the task interface: https://github.com/PrefectHQ/prefect/issues/2619
    i

    itay livni

    2 years ago
    Ok. Cool. So the functionality wont go away
    Chris White

    Chris White

    2 years ago
    yea, correct
    i

    itay livni

    2 years ago
    @Chris White So now that I tried
    task_run_id
    🙂 and
    task_tags
    . I am Still puzzled how to differentiate task runs and cache their unique results. With
    cache_validator=partial_inputs_only(validate_on=['x', 'y']))
    then previous result will be overwritten?
    Chris White

    Chris White

    2 years ago
    Hey itay sorry just seeing this; the
    cache_validator
    interface is currently completely independent from the
    target
    interface, so it doesn’t write any data to disk. We’re hoping to unify the two in a more intuitive way but in the meantime I think using tags is the best way to go (for now).