@Marvin what are the supported keys that can be put in the template string for the persist_storage_key of tasks and flows.

The

persist_storage_key

template string in Prefect allows you to define custom storage keys for results persistence. However, the specific supported keys for this template string aren't explicitly detailed in the documentation excerpts I found. Typically, these keys can include placeholders for dynamic values such as task or flow names, run IDs, etc. To get the exact supported keys, you should refer to the Prefect documentation related to results persistence or the source code if you're comfortable diving into it. You can start by reviewing the results.py file in the Prefect GitHub repository. If you have specific requirements or need further details, feel free to ask!

@Marvin this is the function:

def _format_user_supplied_storage_key(key: str) -> str:
    # Note here we are pinning to task runs since flow runs do not support storage keys
    # yet; we'll need to split logic in the future or have two separate functions
    runtime_vars = {key: getattr(prefect.runtime, key) for key in dir(prefect.runtime)}
    return key.format(**runtime_vars, parameters=prefect.runtime.task_run.parameters)

The function

_format_user_supplied_storage_key

uses placeholders from the

prefect.runtime

module to format the user-supplied storage key. Here’s a breakdown of how it works: 1. Runtime Variables: It collects all attributes from

prefect.runtime

, which likely includes dynamic runtime information like task run IDs, flow run IDs, and other execution context variables. 2. Formatting: The function formats the given

key

string using these runtime variables. It also specifically includes

parameters

from

prefect.runtime.task_run.parameters

, allowing you to use task parameters in your storage key. To understand what specific keys you can use in the template string, you should explore the attributes available in

prefect.runtime

. This might include: - Task run IDs - Flow run IDs - Task/flow names - Parameters passed to the task/flow For more precise details, you would need to explore the

prefect.runtime

module to see what attributes are available. If you need help with this, let me know!

@Marvin can I log in my task the generated uid storage key maybe as an artifact ?

Yes, you can log the generated UID storage key as an artifact in a Prefect task. Prefect provides an interface for creating artifacts, which are pieces of data created by a flow or task run. Here's how you might do it: 1. Logging the Storage Key: You can log the storage key using Prefect's logging capabilities. Prefect uses Python's logging module, so you can simply log the storage key within your task. 2. Creating an Artifact: You can create an artifact using functions provided in the

prefect.artifacts

module. For example, you can use

create_markdown_artifact

or other artifact creation functions based on your needs. Here's a basic example of how you might log a storage key as a Markdown artifact:

python
from prefect import task, flow
from prefect.artifacts import create_markdown_artifact

@task
def log_storage_key(storage_key: str):
    # Log the storage key
    create_markdown_artifact(
        key="storage-key",
        markdown=f"Generated Storage Key: `{storage_key}`"
    )

@flow
def my_flow():
    storage_key = "some-generated-uid"  # This would be your actual storage key
    log_storage_key(storage_key)

my_flow()

In this example,

create_markdown_artifact

is used to log the storage key as a Markdown artifact. You can explore more artifact types and functions in the

prefect.artifacts

module. For more details, you can refer to the artifacts.py file in the Prefect GitHub repository.