Hi, I have a quick question on cached tasks: will ...
# ask-communitya
Alex Papanicolaou
08/15/2020, 6:02 PMHi, I have a quick question on cached tasks: will Prefect skip parts of the graph that are cached and don’t need to be computed so it can jump to only the tasks that do need computation?
n
nicholas
08/15/2020, 10:22 PM
Hi @Alex Papanicolaou - that's correct, cached tasks will only be re-computed if no cached results exist (or they've been invalidate for some reason)
a
Alex Papanicolaou
08/15/2020, 10:24 PMAh, sorry, meant something slightly different (and I know the answer after diving into it). What I meant was that Prefect would be able to pre-check the cache existing and completely removing nodes from consideration in the flow.
Alex Papanicolaou
08/15/2020, 10:24 PMAlex Papanicolaou
08/15/2020, 10:25 PMfor example, if only g didn’t have a cache and I could check that c, d, e, and f had caches, I could load up d and f and cut out c and e entirely (a and b are parameters)
Alex Papanicolaou
08/15/2020, 10:25 PMbut I know this isn’t possible.
Alex Papanicolaou
08/15/2020, 10:25 PMtheoretically it is, but not practically, given how results/caches work.
n
nicholas
08/15/2020, 10:27 PM
Ahh I see what you're saying, sorry for misunderstanding. That's not possible right now.
nicholas
08/15/2020, 10:29 PM
If you have a compelling use case for it I'm sure others do too, so feel free to open up a GitHub discussion 🙂
a
Alex Papanicolaou
08/15/2020, 10:31 PMThanks. Working on something now but it’s tricky and may do enough weird things we’re willing to take on that it would look out of place in the library. Can certainly write it up and see if it provides any inspiration.
👍 1