Simon Gasse10/26/2021, 11:49 AM
files. Is there an automated way in Prefect to clear all data older than a certain date, or do we have to rely on S3 for that?
timestamp and based on that, manually removes the old objects. I honestly believe that using S3 lifecycle policy is the easiest and most reliable way of regularly purging old result files. If you don’t need those results in general, you can turn off checkpointing on a task:
@task(checkpoint=False) def ...
Simon Gasse10/26/2021, 3:15 PM
Simon Gasse10/27/2021, 7:35 AM