I’m using S3 storage for flows running with an ECS Agent. By default the flow’s tasks uses S3Result as outlined here. Is there any way to disable the task results for individual or all tasks in the flow while keeping the S3Storage on the flow?

It sounds like you might want to disable checkpointing, e.g., by setting

PREFECT__FLOWS__CHECKPOINTING

env var to false. I haven't tried this myself, but it's what I figured out from: https://docs.prefect.io/core/concepts/results.html#pipeline-persisted-results

Thanks Amanda! Reading through that page makes me realise I need to rethink my ideas a little here instead of actually disabling the checkpointing. For example, I might actually want to keep the S3Results and instead not pass actual results between tasks but a pointer to the results like uploading a big file to S3 within a task and pass the key for it to downstream tasks.

Hi @Noah Holm! When dealing with large volumes of data, I adopt the strategy you’ve outlined here 💯 . As an FYI, I believe you can remove a result from an individual task in a Flow by specifying

result=None

in your Task decorator.

Nice Dylan 🙂 I did try

@task(result=None)

but it didn’t make any difference. I guess it shouldn’t since that’s the default value so how would it know that I explicitly set it to None and that it wasn’t the default when the flow is evaluated.

Ahh true

Ah awesome! That kind of flag is exactly what I was looking when asking. Ty 🙏