https://prefect.io logo
m

Milly gupta

02/18/2021, 11:47 AM
Hi all, Does Prefect agent persist all task results some where?
a

Amanda Wee

02/18/2021, 12:09 PM
Refer to: https://docs.prefect.io/core/concepts/results.html especially the section on persisting results
z

Zanie

02/18/2021, 3:53 PM
Hi @Milly gupta, it's worth noting that the agent simply deploys runs and nothing else. The question should be: Does the flow run persist task results somewhere? if that doc doesn't help let me know!
m

Milly gupta

02/18/2021, 3:56 PM
Yeah I guess that's what I am trying to find out @Zanie I am looking at Hybrid cloud and what would involve deploying Prefect Agent on our infrastructure?
z

Zanie

02/18/2021, 3:59 PM
Most of the agent's can be started with a simple
prefect agent <type> start
command on a vm or be installed into something like kubernetes depending on what infrastructure you have. They will query Cloud for new runs and when they detect one they'll start it. The run persists data locally or on some form of remote storage (like S3) if requested.
m

Milly gupta

02/18/2021, 4:00 PM
When you say run persists data locally, it's on Prefect agent? right?
z

Zanie

02/18/2021, 4:01 PM
If you are using a
LocalAgent
and
LocalStorage
it will persist data to the machine that the agent is running on, yes.
m

Milly gupta

02/18/2021, 4:01 PM
If a flow is failed and to run it again, the task results needs to be persisted somewhere, right? So if it's on Agent then Agent either need a S3 like storage or a mounted volume
z

Zanie

02/18/2021, 4:02 PM
The agent doesn't know anything about the persistence, is my point.
But yes I think you get the idea.
m

Milly gupta

02/18/2021, 4:39 PM
Yeah thank you. I am just a little confused how and where task results are stored I guess. Want to make sure it's not stored on Prefect Cloud?
z

Zanie

02/18/2021, 4:42 PM
Yep, it's not unless you use a
PrefectResult
explicitly. We don't want to see your data and we're careful not to 🙂
👍 1
5 Views