Krzysztof Nawara
10/09/2020, 8:06 PMprefect.context.map_index
might not be enoughnicholas
Krzysztof Nawara
10/09/2020, 9:58 PMmap_index
incorrect results are going to be read from cache.
Does it make more sense now?nicholas
@task(
cache_key="some_global_key", # to share among mapped children
cache_for=timedelta(days=1),
cache_validator=partial_inputs_only(validate_on=['x', 'y']))
def add(x, y):
return x + y
where you can validate the cache on something like the name of the file that you pass from the upstream task, and any other number of inputsKrzysztof Nawara
10/10/2020, 8:11 AM- state (State): a `Success` state from the last successful Task run that contains the cache
- inputs (dict): a `dict` of inputs that were available on the last
successful run of the cached Task
- parameters (dict): a `dict` of parameters that were available on the
last successful run of the cached Task
nicholas
Krzysztof Nawara
10/12/2020, 6:07 PMnicholas
Krzysztof Nawara
10/14/2020, 7:44 AM