Florent VanDeMoortele
12/23/2025, 1:04 PMMarvin
12/23/2025, 1:04 PMMarvin
12/23/2025, 1:10 PMfrom prefect import flow
from prefect.assets import materialize, add_asset_metadata
@materialize("<tracking://customers/daily>")
def build_customers(date: str):
# Do your compute here...
add_asset_metadata(
"<tracking://customers/daily>",
{"date": date, "rows": 123456, "status": "ok"}
)
@flow
def pipeline(date: str):
build_customers(date)
if __name__ == "__main__":
pipeline("2025-01-01")
This creates an asset in Prefect Cloud at key tracking://customers/daily and attaches your metadata—no file storage required.
2) Track a real file in GCS (you write the file yourself)
from prefect import flow
from prefect.assets import materialize, add_asset_metadata
from google.cloud import storage
@materialize("<gs://my-bucket/datasets/daily/customers.csv>")
def build_customers(date: str):
# Create and upload your data
client = storage.Client()
bucket = client.bucket("my-bucket")
key = f"datasets/daily/customers_{date}.csv"
bucket.blob(key).upload_from_string("id,name\n1,alice")
# Log runtime metadata to the asset in Cloud
add_asset_metadata(
"<gs://my-bucket/datasets/daily/customers.csv>",
{"date": date, "blob": key, "rows": 1}
)
@flow
def pipeline(date: str):
build_customers(date)
if __name__ == "__main__":
pipeline("2025-01-01")
Optional: if you want to actually point the asset key at the specific blob (per-date), you can set the asset key dynamically at call time:
from prefect.assets import materialize
@materialize("<gs://placeholder>") # default; will be overridden
def build_customers(date: str): ...
@flow
def pipeline(date: str):
dynamic = build_customers.with_options(
assets=[f"<gs://my-bucket/datasets/daily/customers_{date}.csv>"]
)
dynamic(date)
Also useful: regular logs to Cloud
from prefect import flow, task
from prefect.logging import get_run_logger
@task
def compute():
logger = get_run_logger()
<http://logger.info|logger.info>("This message appears in Prefect Cloud logs.")
return 42
@flow
def pipeline():
return compute()
Where this shows up in Cloud
- You’ll see your assets in the Assets view with status (succeeded/failed), history, and any metadata you add.
- Using “tracking://…” is perfectly fine when you want observability without storing a file anywhere.
Docs that walk through this
- Assets concepts: Assets concepts
- Assets how-to: Assets how-to
- Advanced assets: Advanced assets
- Logging to Cloud: Logging customization
Since you mentioned you’re on Prefect 2 today and GCP:
- Assets and @materialize are a Prefect 3 feature. To use them, run your flow with Prefect 3 against the same Cloud workspace.
- If you do want to persist files in GCS from Prefect, you can use either google-cloud-storage (as above) or prefect-gcp utilities (e.g., GcsBucket or cloud_storage_upload_blob_from_string).
Would you like a short, ready-to-run example wired to your GCS bucket name and project, or are you planning to use tracking-only assets first?Florent VanDeMoortele
12/23/2025, 1:12 PMMarvin
12/23/2025, 1:13 PMMarvin
12/23/2025, 1:13 PMfrom prefect import flow
from prefect.assets import materialize, add_asset_metadata
@materialize("<tracking://customers/daily>")
def build_customers(date: str):
# Do work...
add_asset_metadata("<tracking://customers/daily>", {"date": date, "rows": 123456})
@flow
def pipeline(date: str):
build_customers(date)
- create_markdown_artifact (UI report)
from prefect import flow
from prefect.artifacts import create_markdown_artifact
@flow
def pipeline():
md = "# Daily Summary\n\n- Rows processed: 123,456\n- Status: ✅"
create_markdown_artifact(key="daily-summary", markdown=md, description="Run summary")
You can use both together: materialize to log/update the asset and lineage, and create_markdown_artifact to attach a rendered report for humans.
Docs
- Assets: Assets concepts, Assets how-to, Advanced assets
- Artifacts: Artifacts how-to, Artifacts concepts
Since you’re on Prefect 2 Cloud today: materialize is available in Prefect 3. If you stay on 2.x, you can still use create_markdown_artifact for reporting; upgrading to 3 unlocks the Assets graph and @materialize.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by