https://prefect.io logo
Title
s

Soami Charan

05/23/2023, 9:14 AM
Hi, Is there is any way that task cache can be store on S3 instead Local storage?
j

Justin Trautmann

05/23/2023, 12:34 PM
i think this can be achieved by storing the task's result on s3. just create an S3 storage block via the UI or python and reference it by name in the result_storage argument of your task
from prefect import task
from prefect.filesystems import S3

@task(result_storage=S3.load("my_s3_block_name"))
def my_task():
    pass
s

Soami Charan

05/23/2023, 2:23 PM
What about task cache data?
j

Justin Trautmann

05/23/2023, 3:33 PM
what do you mean with cache data? as far as i understand it, the cache is build on top of the result storage. when the prefect server sees a cache hit, the task is not executed but instead the following tasks will retrieve the result from the result storage.