https://prefect.io logo
Powered by
Title
a

Avi A

05/27/2020, 12:35 PM
Question regarding the 
Result
we longed for: I see that the default is to cache the results per task, and it’s possible to cache per day/today/tomorrow when using the target formatting. Is there a behavior similar to 
cache_validator
, allowing us to cache the results per set of attributes?
👀 1
advanced: can we cache the results based on the input to the upsteam tasks? example: 
@task
def A(p: int):
  return some_object

@task
def B(p: int):
  return some_object

with Flow() as flow:
  p = Parameter('p')
  a = A(p)
  b = B(a)
can I cache the result of 
B
based on the value of the input parameter 
p
(not 
a
), without specifically mentioning it in task 
B
? meaning having it read from the upsteam?
n

nicholas

05/27/2020, 12:51 PM
Hi @Avi A - we have an open ticket that'll enable the behavior you've described; I'm sure your input is welcome there if you have ideas of how you might like to see it implemented (or just an emoji reaction to let them know you're also interested in seeing this)!
a

Avi A

05/27/2020, 12:53 PM
thanks @nicholas! Related: if I don’t supply 
target
, it seems as if the task is not cached at all. I mean, it is cached, but since the exact timestamp when the task was run (not even including its name), then this cache is practically never read 😞
I mean, it’s better to use the deprecated 
cache_for
n

nicholas

05/27/2020, 1:10 PM
Interesting @Avi A - is there any reason you might not want to use a result target with some sort of result templating?
a

Avi A

05/27/2020, 1:11 PM
well, because I can’t use the task parameters in the template
I see I can use the flow parameters in the template, because they appear in the context (given they were provided to the flow, and not using the defaults)
n

nicholas

05/27/2020, 1:22 PM
Ah ok! I think you can use task parameters in the in the location template, since they're available in context. Let me confirm that for you quickly though
a

Avi A

05/27/2020, 1:24 PM
I can
but only if they were provided explicitly (i.e. not used the defaults)
also, I’d like to cache a specific task based on the parameters it recursively depends on, and not necessarily on all the params
n

nicholas

05/27/2020, 1:31 PM
Seems like a good use case @Avi A - I'd encourage you to add your thoughts to that issue above, they'll be valuable in deciding the direction/implementation of that ticket
👍 1
#prefect-communityJoin Slack
Powered by