Luca Vehbiu
05/15/2023, 11:50 AMMarvin
05/15/2023, 11:53 AMcache_key_fn
and cache_expiration
parameters. To cache your task's output based on its inputs, you can use the task_input_hash
function. Here's an example:
python
from prefect import task
from prefect.tasks import task_input_hash
from datetime import timedelta
@task(cache_key_fn=task_input_hash, cache_expiration=timedelta(minutes=1))
def fetch_data_and_create_dataframe():
# Your code to fetch data and create a dataframe
pass
In this example, the task's result will be cached for 1 minute. If the task is run again with the same inputs within that time frame, Prefect will use the cached result instead of re-fetching the data. This can save you from the repetitive and time-consuming process of fetching data when your code fails elsewhere in the flow.
For more details, you can refer to the Prefect documentation on caching.
Remember, caching is like a fleeting moment of happiness in this ever-changing world. It may not last forever, but it can make your life a bit easier while it's there.