https://prefect.io logo
d

Daniel Nussbaum

12/04/2020, 7:18 PM
Apologies if this is a basic question -- we are considering using Prefect Cloud for our ETL tasks. I cannot find specifically what data is sent from Prefect Server to Prefect Cloud -- is there a document outlining the data being sent and security standards / policies more generally? I'd appreciate anyone who can point me in the right direction here!
n

nicholas

12/04/2020, 7:24 PM
Hi @Daniel Nussbaum - to clarify, are you specifically talking about Telemetry data sent from Prefect Server or what data is sent to Cloud when you register a flow from Prefect Core?
d

Daniel Nussbaum

12/04/2020, 7:26 PM
Both!
n

nicholas

12/04/2020, 7:27 PM
I can say definitively that no data that isn't about the structure of your flow is sent to Prefect that you don't explicitly log. When you register a flow with Prefect, the dependency structure and peripheries (storage, environment, schedules etc) are serialized, so that Prefect knows what you want to happen and where. However, the internals of task and data that you pass between tasks are never sent to Prefect.
d

Daniel Nussbaum

12/04/2020, 7:29 PM
Got it - so no actual data ever makes it to Prefect Cloud except for information that there is a flow scheduled (and metadata about that flow which you choose to send).
👍 1
n

nicholas

12/04/2020, 7:32 PM
That's correct! In addition, if you're running Prefect Server, even metadata is never sent to Prefect, those are completely separate applications. Telemetry (which can be turned off!) in Prefect Server consists of an anonymous ID and heartbeats that we use to understand how often Prefect Server is being deployed.
Here’s a doc that goes into the details of data flow and Cloud
d

Daniel Nussbaum

12/04/2020, 7:34 PM
Thanks for the clarifications here! I’ll share this with our security team.
😄 1