@Marvin What would be the closest to a namedtempfile for storing results? I don't need it beyond the duration of a flow, I don't want it cached. The LocalFileStorage block is kinda clunky to work with with respect to this use-case

It seems you're looking for a way to handle temporary result storage within a Prefect flow, akin to a named temporary file, but without the permanence or caching that comes with some of Prefect's storage options like

LocalFileStorage

. In Prefect 3.x, task results are stored in memory by default, which might suit your use case if the data is small and only needed for the duration of the flow. However, if you need to store the results on disk temporarily, you might want to consider using the

WritableFileSystem

interface to write and read data directly within your flow code. This allows you to handle files in a custom manner, and you can ensure they are deleted at the end of the flow. Here's a basic example of how you might implement this:

python
from prefect import flow, task
import tempfile
import os

@task
def write_to_temp_file(data):
    with tempfile.NamedTemporaryFile(delete=False) as tmp_file:
        tmp_file.write(data.encode())
        return tmp_file.name

@task
def read_and_delete_temp_file(file_path):
    with open(file_path, 'r') as file:
        content = file.read()
    os.remove(file_path)
    return content

@flow
def temp_file_flow(data):
    file_path = write_to_temp_file(data)
    result = read_and_delete_temp_file(file_path)
    print(result)

temp_file_flow("Hello, Prefect!")

This approach uses Python's

tempfile

module to create a temporary file, which is then deleted after use. It's a straightforward way to handle ephemeral storage needs within a flow. For more information, you can explore Prefect's results module and filesystems module. If you need to handle larger data or specific storage requirements, integrating with cloud storage providers via Prefect's storage blocks might be an option, albeit with more complexity.