https://prefect.io logo
Title
c

CA Lee

09/20/2020, 9:54 AM
Hi, I have 2 tasks - first one fetches data, second one inserts the fetched data into a database The tasks will be run on a TimeInterval schedule. I have cached the first tasks's output. How do I avoid running the second task if the cached output from the first task is still valid i.e. cache age is still < 1 day?
@task(cache_for=datetime.timedelta(days=1))
def fetch_data():
    return fetched_data

@task
def insert_into_database(fetched_data):
    fetched_data.to_sql('table_name', con=db.engine)
Hello @nicholas, would you happen to have any insight on this?
k

Kyle Moon-Wright

09/21/2020, 5:58 PM
Hey @CA Lee, I think you can approach this by implementing a custom trigger on the second task that explicitly checks the cache expiration attribute of the state of 
fetch_data
, then executes
insert_into_database
. I don't have a snippet, but I'll try to write/find an example of a custom trigger.