In Prefect 3.0.3, when you don't explicitly configure

key_storage

with a cache policy, cached results are stored by default in your local filesystem at:

~/.prefect/storage/

This is the default local storage location that Prefect uses. However, it's important to note that this local storage means the cache will only be available on the machine where the flow was executed. If you need to share cached results across different machines or workers, you should configure a storage block (like S3, Azure, or GCS) using the

result_storage

parameter in your task decorator. You can find more details about caching configuration in the Prefect documentation on caching. Let me know if you'd like to learn more about setting up shared storage for cached results or have any other questions!

Based on the research and relevant GitHub issues, here's what happens with cache storage in a Docker container: 1. When running in a Docker container, the default cache location is still

~/.prefect/storage/

, but this path exists INSIDE the container's filesystem. This means: - The cache is isolated to that specific container - The cache will be lost when the container is removed - If you're running multiple containers, each will have its own separate cache 2. Important Implications: - If your container restarts, the cache will be lost unless you've mounted a volume to persist the

.prefect

directory - Different container instances won't share the cache by default - Tasks running in different containers won't be able to access each other's cached results 3. Best Practices for Docker Environments: - For persistent caching across container restarts, you should either: a. Mount a volume to persist the

.prefect

directory:

yaml
     volumes:
       - ./prefect-data:/root/.prefect

b. Use a remote storage block (recommended) like S3, GCS, or Azure for cache storage:

python
     @task(cache_key_fn=task_input_hash, 
          result_storage=S3Bucket.load("my-bucket"))
     def my_task():
         ...

4. Example of configuring remote storage for caching:

python
from prefect.filesystems import S3
from prefect.tasks import task

# Configure S3 storage
s3_block = S3(bucket_path="my-cache-bucket")
s3_block.save("cache-storage")

@task(
    cache_key_fn=task_input_hash,
    result_storage="s3/cache-storage"  # Reference the storage block
)
def my_cached_task():
    ...

This approach ensures your cached results are available across container restarts and can be shared between different containers or workers. Let me know if you'd like more specific examples of setting up persistent cache storage for your Docker environment!