Simon Gasse10/26/2021, 11:49 AM
files. Is there an automated way in Prefect to clear all data older than a certain date, or do we have to rely on S3 for that?
Anna Geller10/26/2021, 11:58 AM
timestamp and based on that, manually removes the old objects. I honestly believe that using S3 lifecycle policy is the easiest and most reliable way of regularly purging old result files. If you don’t need those results in general, you can turn off checkpointing on a task:
@task(checkpoint=False) def ...
Kevin Kho10/26/2021, 1:55 PM
Simon Gasse10/26/2021, 3:15 PM
Kevin Kho10/26/2021, 3:20 PM
Simon Gasse10/27/2021, 7:35 AM