Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hii all. We're about to have Christmas, so no rush in answering this. My question is: what is the best way to package artifacts (a parquet file) and environment variables for a Prefect Flow to work as intended?

Hi Bernardo! How do you plan to run your flows?

The best way might differ if you're running flows as processes on a single machine vs. running them in a Docker container vs running them as an ECS task.

Also, how large do you expect the Parquet files to be?

Hi :slightly_smiling_face: Came back on the 26th to respect holidays.

For one flow, these are the files

The parquet file in question is under 1MB

ideally, I can run the flow using a codebase, like a git repo

I suppose it is easier to run in a docker container