<@ULVA73B9P> If artifacts are stored on prefect cl...
# ask-marvin
l
@Marvin If artifacts are stored on prefect cloud, do they have an expiration date? For how long are they kept around? And is there a limit on how many artifacts we can have at any given time?
m
thought for 416.1 seconds
Short answer: - Expiration: There is no per‑artifact TTL in Prefect Cloud today. Artifacts do not auto‑expire; they’re retained until you delete them (or the workspace is deleted). - How long they’re kept: Indefinitely by default. - Limits: There’s no published hard cap on the number of artifacts per workspace. API rate limits apply, and very large volumes may impact UI/API performance. If you expect very large volumes (e.g., millions), let us know and we can advise. Notes and references - Artifacts overview (Prefect 3.x): Artifacts — concepts - CLI for inspecting/deleting artifacts: Prefect CLI reference (see the
prefect artifact
commands below) Managing/cleaning up artifacts - Overwrite instead of accumulating: set a stable
key
when creating artifacts so the “latest” for that key is updated instead of creating a new row each time. - Delete with the CLI:
Copy code
prefect artifact ls --limit 100
  prefect artifact inspect <artifact-id>
  prefect artifact delete <artifact-id>
- Delete with the Python client (example async cleanup by age):
Copy code
import asyncio
  from datetime import datetime, timedelta, timezone
  from prefect.client.orchestration import get_client

  async def delete_old_artifacts(days=30, batch_size=1000):
      cutoff = datetime.now(timezone.utc) - timedelta(days=days)
      async with get_client() as client:
          offset = 0
          while True:
              artifacts = await client.read_artifacts(limit=batch_size, offset=offset)
              if not artifacts:
                  break
              for a in artifacts:
                  # delete based on your criteria; many users use a.created
                  if getattr(a, "created", None) and a.created < cutoff:
                      await client.delete_artifact(a.id)
              offset += len(artifacts)

  asyncio.run(delete_old_artifacts(30))
A couple of clarifications to help tailor guidance: - Which Prefect Cloud plan are you on (Hobby/Team/Enterprise)? Some organizations have custom data retention policies. - Roughly how many artifacts do you expect to keep and for how long? If you’re planning for very high volume, we can suggest patterns (e.g., keying/versioning, periodic pruning flows, or offloading heavy payloads to external storage).
l
@Marvin we are on Enterprise plan. Could you explain further what you mean by keying/versioning in the context of artifacts? Is it possible "append" to an existing artifact? i.e. fetch an existing artifact by key and then add more data to it based on some kind of override flag.
m
thought for 260.7 seconds
Great — thanks for the context. How keying/versioning works for artifacts - Key is a stable string you assign when creating an artifact (e.g.,
key="daily-report"
). - Every call to
create_*_artifact
creates a new artifact row. If you reuse the same key, Prefect groups those rows into a collection for that key and marks one as the “latest.” - In the UI (and via the CLI), you’ll typically see the latest artifact per key by default; you can also view the full history for that key. Can you “append” to an artifact? - There’s no generic “append” or “overwrite in-place” API for artifacts like markdown/table/image/link today. - The one exception is progress artifacts: you can update progress via
update_progress_artifact(artifact_id=..., progress=...)
. - For other artifact types, “append” is implemented by: 1) fetching the latest artifact for a given key, 2) merging your new content with the existing content in your code, 3) creating a new artifact with the same key — which becomes the new “latest.” Practical patterns 1) Create an artifact with a stable key (new version each time)
Copy code
from prefect import flow
from prefect.artifacts import create_markdown_artifact

@flow
def report():
    create_markdown_artifact(
        markdown="# My Daily Report\nStarted...",
        key="daily-report",
        description="Daily status report"
    )

report()
2) Fetch the latest artifact by key, modify, and create a new “version” (append-like)
Copy code
import asyncio
from prefect.client.orchestration import get_client
from prefect.client.schemas.filters import ArtifactCollectionFilter, ArtifactCollectionFilterKey
from prefect.artifacts import acreate_markdown_artifact

async def append_to_markdown(key: str, extra: str):
    async with get_client() as client:
        cols = await client.read_latest_artifacts(
            artifacts=ArtifactCollectionFilter(
                key=ArtifactCollectionFilterKey(any_=[key])
            ),
            limit=1
        )
        if not cols:
            # nothing exists yet; create the first version
            await acreate_markdown_artifact(markdown=extra, key=key)
            return

        col = cols[0]
        # Fetch the latest artifact record by id
        latest = await client.read_artifact(col.latest_id)
        # Merge existing markdown with new content
        current_md = latest.data.get("markdown", "")
        new_md = current_md + "\n" + extra
        # Create a new artifact with the same key (becomes the latest)
        await acreate_markdown_artifact(markdown=new_md, key=key)

asyncio.run(append_to_markdown("daily-report", "…and more details."))
3) Update a progress artifact in-place
Copy code
from prefect.artifacts import create_progress_artifact, update_progress_artifact

artifact_id = create_progress_artifact(progress=0.1, description="Starting job...")
update_progress_artifact(artifact_id, progress=0.5, description="Halfway there")
update_progress_artifact(artifact_id, progress=1.0, description="Done")
4) CLI tips (latest vs history) - By default,
prefect artifact ls
shows the latest per key; add
--all
to see the full version history.
Copy code
prefect artifact ls
prefect artifact ls --all
prefect artifact inspect <artifact-id>
prefect artifact delete <artifact-id>
Docs for reference - Artifacts concepts (Prefect 3.x): Artifacts — concepts - CLI reference: Prefect CLI reference (see
prefect artifact
) Enterprise note - There isn’t a built-in TTL for artifacts. If you want a retention window, the common pattern is to run a scheduled cleanup flow that deletes older artifact versions per key (keeping only the latest N or those newer than X days). If you’d like, I can share a ready-to-use cleanup flow template or work with you on org-level options.