@Marvin Can Prefect task parameters be large, like an entire df, or is there a penalty to this?

Passing large objects like an entire DataFrame as task parameters can cause performance issues due to introspection time. Prefect must inspect each element, which can be time-consuming. To mitigate this, you can use the

_quote_

utility to reduce inspection time, though it will disable task dependency tracking for the wrapped object. References: - Introspection time issue: https://github.com/PrefectHQ/prefect/issues/11480 - Large data handling guide: https://docs.prefect.io/latest/guides/big-data/