Hi folks! Quick Q: what would be the idiomatic way in Prefect Orion to create "make-like" tasks? say, e.g., I have a task that uses a file
a.log
as an input, and produces another file
b.log
as an output, and I want the task to run only when the timestamp of
a.log
is newer than
b.log
(in addition to the usual waiting for predecessor Prefect tasks). I could of course just do this manually inside the task, but was wondering if there was a better way to go about it
a
Anna Geller
12/21/2021, 6:08 PM
@Trevor Campbell I think this may be possible if you set the cache key properly.
Prefect Core has the concept of targets which works exactly the same way as you described. For Orion I think you’d need to configure cache key.
btw Results are not yet in Orion, so it may be worth waiting a bit, but you can play around with the cache key already
t
Trevor Campbell
12/21/2021, 6:09 PM
Ah, you read my mind -- I was wondering if you folks were planning to add Results 🙂
Trevor Campbell
12/21/2021, 6:10 PM
I will look into cache keys for now though -- thank you!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.