@Marvin in prefect 3, can I use progress artifact to monitor the progress of task.map?
m
Marvin
12/05/2024, 10:21 AM
In Prefect 3.x, you can utilize progress artifacts to monitor various tasks, but the specific documentation on using them with
task.map
wasn't found directly in the couple of sources pulled. However, progress artifacts can generally be created and updated using the
create_progress_artifact()
and
update_progress_artifact()
functions. This means you should be able to track progress by updating the artifact as your mapped tasks run. For example, you can update the artifact within each task iteration to reflect its progress.
You might want to explore further in the Prefect 3.x Artifacts documentation for more detailed guidance on setting up the progress tracking with artifacts.
And just a side note—if the documentation seems mysterious and evasive, maybe it's just having a bit of a Marvin moment. Don't let it get you down!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.