https://prefect.io logo
l

Lior Barak

09/18/2023, 2:18 PM
hey friends! anyone know how to delete artifacts with code? I only see ways to create
1
j

Jake Kaplan

09/18/2023, 2:59 PM
Hi, you can delete artifacts using the client or hitting the api directly
Copy code
from prefect.client.orchestration import get_client

async def main():
    async with get_client() as client:
        await client.delete_artifact(artifact_id=...)
🙌 2
l

Lior Barak

09/18/2023, 3:34 PM
so if i'm running a flow inside an agent, is this common practice? or am I misusing artifacts?
j

Jake Kaplan

09/18/2023, 4:02 PM
Sorry not sure I understand. Are you asking if it's normal practice to delete an artifact inside of a flow directly? If so you certainly can! It really depends how you're using artifacts
s

Serina

09/18/2023, 7:17 PM
@Lior Barak, I am curious about the use case for deleting them? 🙂 I think one common usage for artifacts is that they are byproducts of a flow run. provided you give the artifact the same
key
(think s3 buckets), you’re going to see the lineage of the artifact over time (how the data in the table changes, for example) in your Artifacts page.
l

Lior Barak

09/18/2023, 7:50 PM
@Serina this is actually pretty specific so bear with me: first, this is mostly for internal testing so it's an edge case, and the idea is to not use minimal external infra to achieve my goal I have a use case where the same deployment (and the same flow) is called multiple times some testing script/manually. the expected behaviour is that the flow will reach the same result don't know if this matters but the flows result is a large-ish (1MB) json file what I want is a way for the tester/script to know that the flow is finished and to be able to download the resulting json file usually this is not a problem and the flow posts the result to some queue, in the testing case we have less infrastructure and I was thinking of letting the tester script poll the prefect API for the result. for that I was thinking to have an artifact saved when the flow finished ("flow-finished-{some-id}"), and the next time that flow starts deleting that artifact so past tests won't influence current tests this seems a little strange to me that the SDK doesn't have an Artifact deletion option so I was thinking maybe this is not the intended use for artifacts 🤔 (oh and if it matters i'm self hosting on K8s at the moment and not using prefect cloud for event driven stuff)
s

Serina

09/18/2023, 9:52 PM
There is a way to delete artifacts via the CLI: https://docs.prefect.io/latest/concepts/artifacts/#deleting-artifacts I’m not totally certain I understand exactly, but it sounds like maybe state change hooks could be useful here? You can call anything you like when a flow completes: https://docs.prefect.io/latest/concepts/states/#state-change-hooks
l

Lior Barak

09/19/2023, 7:13 AM
@Serina i'm gonna try and use @Jake Kaplan’s suggestion and see how it fits. the CLI/change hooks isn't too useful for this specific use case
s

Serina

09/19/2023, 2:48 PM
It would be a way to do this anyway:
what I want is a way for the tester/script to know that the flow is finished and to be able to download the resulting json file
but yeah, Prefect is pretty extensible! :)
2 Views