https://prefect.io logo
#prefect-community
Title
# prefect-community
s

Steph Clacksman

10/05/2022, 10:57 AM
Is there a limit on data size for params for a subflow? I have a flow which passes two dataframes to a subflow, and one of the dataframes causes an error to be thrown:
TypeError: Dict key must be str
but the other doesn't. The one causing the error does have ~1 million rows though.
k

Khuyen Tran

10/05/2022, 1:35 PM
Hmm. What is the difference between the dataframe that causes an error and the dataframe that doesn’t cause the error?
s

Steph Clacksman

10/11/2022, 8:11 AM
I have made a minimal reproducible example of a dataframe throwing a typeerror when passing to a subflow:
Copy code
import pandas as pd
from prefect import flow


@flow
def test_flow() -> None:
    df = pd.DataFrame({"ID": ["123456789", "223456789"]})
    test_subflow(df)


@flow(validate_parameters=False)
def test_subflow(df: pd.DataFrame) -> None:
    print(df)


if __name__ == "__main__":
    test_flow()
k

Khuyen Tran

10/13/2022, 3:33 PM
It seems to be the same issue as this one so we are working on making it work