Laxmi
09/01/2025, 12:45 PMMarvin
09/01/2025, 12:45 PMMarvin
09/01/2025, 12:52 PMprefect artifact
commands below)
Managing/cleaning up artifacts
- Overwrite instead of accumulating: set a stable key
when creating artifacts so the “latest” for that key is updated instead of creating a new row each time.
- Delete with the CLI:
prefect artifact ls --limit 100
prefect artifact inspect <artifact-id>
prefect artifact delete <artifact-id>
- Delete with the Python client (example async cleanup by age):
import asyncio
from datetime import datetime, timedelta, timezone
from prefect.client.orchestration import get_client
async def delete_old_artifacts(days=30, batch_size=1000):
cutoff = datetime.now(timezone.utc) - timedelta(days=days)
async with get_client() as client:
offset = 0
while True:
artifacts = await client.read_artifacts(limit=batch_size, offset=offset)
if not artifacts:
break
for a in artifacts:
# delete based on your criteria; many users use a.created
if getattr(a, "created", None) and a.created < cutoff:
await client.delete_artifact(a.id)
offset += len(artifacts)
asyncio.run(delete_old_artifacts(30))
A couple of clarifications to help tailor guidance:
- Which Prefect Cloud plan are you on (Hobby/Team/Enterprise)? Some organizations have custom data retention policies.
- Roughly how many artifacts do you expect to keep and for how long? If you’re planning for very high volume, we can suggest patterns (e.g., keying/versioning, periodic pruning flows, or offloading heavy payloads to external storage).Laxmi
09/02/2025, 10:29 AMMarvin
09/02/2025, 10:29 AMMarvin
09/02/2025, 10:33 AMkey="daily-report"
).
- Every call to create_*_artifact
creates a new artifact row. If you reuse the same key, Prefect groups those rows into a collection for that key and marks one as the “latest.”
- In the UI (and via the CLI), you’ll typically see the latest artifact per key by default; you can also view the full history for that key.
Can you “append” to an artifact?
- There’s no generic “append” or “overwrite in-place” API for artifacts like markdown/table/image/link today.
- The one exception is progress artifacts: you can update progress via update_progress_artifact(artifact_id=..., progress=...)
.
- For other artifact types, “append” is implemented by:
1) fetching the latest artifact for a given key,
2) merging your new content with the existing content in your code,
3) creating a new artifact with the same key — which becomes the new “latest.”
Practical patterns
1) Create an artifact with a stable key (new version each time)
from prefect import flow
from prefect.artifacts import create_markdown_artifact
@flow
def report():
create_markdown_artifact(
markdown="# My Daily Report\nStarted...",
key="daily-report",
description="Daily status report"
)
report()
2) Fetch the latest artifact by key, modify, and create a new “version” (append-like)
import asyncio
from prefect.client.orchestration import get_client
from prefect.client.schemas.filters import ArtifactCollectionFilter, ArtifactCollectionFilterKey
from prefect.artifacts import acreate_markdown_artifact
async def append_to_markdown(key: str, extra: str):
async with get_client() as client:
cols = await client.read_latest_artifacts(
artifacts=ArtifactCollectionFilter(
key=ArtifactCollectionFilterKey(any_=[key])
),
limit=1
)
if not cols:
# nothing exists yet; create the first version
await acreate_markdown_artifact(markdown=extra, key=key)
return
col = cols[0]
# Fetch the latest artifact record by id
latest = await client.read_artifact(col.latest_id)
# Merge existing markdown with new content
current_md = latest.data.get("markdown", "")
new_md = current_md + "\n" + extra
# Create a new artifact with the same key (becomes the latest)
await acreate_markdown_artifact(markdown=new_md, key=key)
asyncio.run(append_to_markdown("daily-report", "…and more details."))
3) Update a progress artifact in-place
from prefect.artifacts import create_progress_artifact, update_progress_artifact
artifact_id = create_progress_artifact(progress=0.1, description="Starting job...")
update_progress_artifact(artifact_id, progress=0.5, description="Halfway there")
update_progress_artifact(artifact_id, progress=1.0, description="Done")
4) CLI tips (latest vs history)
- By default, prefect artifact ls
shows the latest per key; add --all
to see the full version history.
prefect artifact ls
prefect artifact ls --all
prefect artifact inspect <artifact-id>
prefect artifact delete <artifact-id>
Docs for reference
- Artifacts concepts (Prefect 3.x): Artifacts — concepts
- CLI reference: Prefect CLI reference (see prefect artifact
)
Enterprise note
- There isn’t a built-in TTL for artifacts. If you want a retention window, the common pattern is to run a scheduled cleanup flow that deletes older artifact versions per key (keeping only the latest N or those newer than X days). If you’d like, I can share a ready-to-use cleanup flow template or work with you on org-level options.