Apologies if this is a basic question -- we are considering using Prefect Cloud for our ETL tasks.
I cannot find specifically what data is sent from Prefect Server to Prefect Cloud -- is there a document outlining the data being sent and security standards / policies more generally?
I'd appreciate anyone who can point me in the right direction here!
12/04/2020, 7:24 PM
Hi @Daniel Nussbaum - to clarify, are you specifically talking about Telemetry data sent from Prefect Server or what data is sent to Cloud when you register a flow from Prefect Core?
12/04/2020, 7:26 PM
12/04/2020, 7:27 PM
I can say definitively that no data that isn't about the structure of your flow is sent to Prefect that you don't explicitly log. When you register a flow with Prefect, the dependency structure and peripheries (storage, environment, schedules etc) are serialized, so that Prefect knows what you want to happen and where. However, the internals of task and data that you pass between tasks are never sent to Prefect.
12/04/2020, 7:29 PM
Got it - so no actual data ever makes it to Prefect Cloud except for information that there is a flow scheduled (and metadata about that flow which you choose to send).
12/04/2020, 7:32 PM
That's correct! In addition, if you're running Prefect Server, even metadata is never sent to Prefect, those are completely separate applications. Telemetry (which can be turned off!) in Prefect Server consists of an anonymous ID and heartbeats that we use to understand how often Prefect Server is being deployed.