Brady Mitchell
07/11/2025, 11:07 PMMarvin
07/11/2025, 11:07 PMMarvin
07/11/2025, 11:07 PMpython
from prefect.tasks import task
from prefect.utilities.caching import cache_key_from_task_source
@task(cache_policy={
"key_fn": cache_key_from_task_source,
"key_storage": "/path/to/your/cache/directory"
})
def my_task():
# your task code here
pass
2. For remote storage, you can use storage blocks like S3:
python
from prefect.tasks import task
from prefect.blocks.system import S3Bucket
s3_block = S3Bucket.load("my-block-name")
@task(cache_policy={
"key_fn": cache_key_from_task_source,
"key_storage": s3_block
})
def my_task():
# your task code here
pass
The key_storage
parameter determines where your cache files will be stored. If you don't specify it, Prefect will use a default local directory.
You can find more details about caching configuration in the Prefect documentation.Brady Mitchell
07/11/2025, 11:14 PMMarvin
07/11/2025, 11:14 PMMarvin
07/11/2025, 11:14 PMbash
# For local storage path
export PREFECT_LOCAL_STORAGE_PATH="/path/to/your/results"
# For a different storage block
export PREFECT_DEFAULT_RESULT_STORAGE_BLOCK="s3/my-block"
2. Per Task/Flow Configuration
Specify storage directly on tasks or flows:
python
from prefect import task, flow
from prefect.blocks.system import S3Bucket
# Using local storage
@task(result_storage="/path/to/storage")
def my_task():
return "some result"
# Using a storage block
s3_block = S3Bucket.load("my-block")
@task(result_storage=s3_block)
def another_task():
return "some result"
3. Using Python Configuration
Configure through Prefect's settings:
python
from prefect.settings import Settings
settings = Settings(
default_result_storage_block="s3/my-block",
# or for local storage:
local_storage_path="/path/to/storage"
)
The precedence order is:
1. Task/flow-level configuration
2. Environment variables
3. Default settings
You can find more details about result storage configuration in the Prefect documentation.
Remember that if you're using local storage, make sure the specified directory exists and is writable by the process running your flows.