Short answer: - No, task arguments do not have to be JSON-serializable. For caching, Prefect computes a cache key by trying to JSON-serialize the inputs and falling back to a pickle-based hash when needed. - For result persistence (e.g., to S3), outputs must be serializable by the configured result serializer. The default in Prefect 3.x is Pickle, so outputs need to be pickle-serializable. What this means for pandas: - Passing a pandas DataFrame as a task argument is fine; Prefect can hash it for caching. - Returning a pandas DataFrame is also fine with the default Pickle serializer when you persist results to S3. - If you explicitly switch to a JSON result serializer, convert DataFrames to a JSON-friendly form (e.g.,

df.to_dict(orient="records")

) before returning. Example:

import pandas as pd
from prefect import task, flow
from prefect.cache_policies import INPUTS

@task(
    cache_policy=INPUTS,            # cache based on inputs
    persist_result=True,            # write result to storage
    result_storage="s3-bucket/my-s3-block"  # your S3 block
    # result_serializer="pickle"    # default; explicit for clarity if you want
)
def summarize(df: pd.DataFrame) -> pd.DataFrame:
    return df.describe()

@flow
def main():
    df = pd.DataFrame({"a": [1, 2, 3]})
    return summarize(df)

if __name__ == "__main__":
    main()

If you prefer JSON results:

@task(persist_result=True, result_storage="s3-bucket/my-s3-block", result_serializer="json")
def summarize_json(df: pd.DataFrame):
    summary = df.describe()
    return summary.to_dict()

Notes: - Objects that cannot be pickled (e.g., open file handles, certain locks) won’t work for caching or result persistence. - For very large DataFrames, consider persisting them as Parquet/CSV to S3 yourself and returning a reference (path) from the task.