Adam
12/29/2019, 11:21 PMTarget
for outputs, and if that Target exists (you get to define “exists”), the task won’t run and will return the already-completed result. Is there something similar for Prefect, or is this something I would have to implement manually? Cacheing seems like it would work, but does that cache persist across runs, or if different `Flow`s are using the same Task
?
As a concrete example - I’m writing a single row per day into a psql database. The day is defined by a Parameter("date")
. If I run my Flow repeatedly, it keeps inserting rows for the same date. Instead, I’d like to check for existence of the row with that date and return with the appropriate State (probably Success
?).Chris White
12/29/2019, 11:49 PMcache_key
keyword argument to tasks (more on this here: https://docs.prefect.io/core/concepts/persistence.html#persistence-and-caching)
For your example, you would use the all_parameters
or partial_parameters_only
cache validator on whatever task performs the write.
Let me know if you have any follow-up questions or need any clarifications!Adam
12/29/2019, 11:55 PMChris White
12/29/2019, 11:58 PMAdam
12/30/2019, 12:46 AM