Prefect 1 0 Is it bad practice to pass large amounts of data

Question

Prefect 1 0 Is it bad practice to pass large amounts of data via Parameters I have a Flow `A` I m using as a flow of flows `A` goes and downloads a bunch of data from S3 I want to pass this data say 500MB of JSON to a child flow `B` via Parameter

Zanie · Answer

Yes definitely that data would be written to our database and needs to go over the network

Zanie · Answer

I presume that would exceed our limits and be denied I d pass the data via a reference to the S3 file instead

Anders Segerberg · Answer

thank you for the context

Anders Segerberg · Answer

Oh well we aren t using Prefect cloud

Anders Segerberg · Answer

Would the same thing apply with our self managed prefect server parameters are written to the postgres DB

Kyle McChesney · Answer

yeah we have 1 0 self managed `parameters` is a column of the `flow run` table and includes the actual values

Anders Segerberg · Answer

thank you