Matthew Wiseman
07/07/2025, 1:20 AMMarvin
07/07/2025, 1:20 AMMarvin
07/07/2025, 1:22 AMpython
from prefect import flow, task
from prefect.assets import Asset
from prefect.utilities.annotations import materialize
@task
def determine_asset_key() -> str:
# Your runtime logic to determine the key
return f"dynamic-key-{some_runtime_value}"
@flow
def my_flow():
# Get the dynamic key
key = determine_asset_key()
# Create and return the result
result = create_asset_with_dynamic_key(key)
return result
@materialize
def create_asset_with_dynamic_key(key: str):
# Your asset creation logic here
data = {"some": "data"}
# Return both the data and the asset metadata
return data, Asset(key=key)
A few key points about this approach:
1. The @materialize
decorator is used to mark the function that will create the asset
2. The function returns a tuple containing:
- The actual data/content of the asset
- An Asset
instance with the dynamic key
3. You can also include additional metadata with the asset if needed:
python
return data, Asset(
key=key,
description="Dynamically generated asset",
url="<https://your-storage-location.com/path>"
)
This pattern gives you full flexibility to determine the asset key at runtime while still maintaining the benefits of Prefect's asset tracking and lineage features.
For more details about working with assets, you can refer to the Prefect Assets documentation.Chris White
Matthew Wiseman
07/07/2025, 1:51 AM