Hi, is there a way to pass in Pandas DataFrames as...
# prefect-cloud
m
Hi, is there a way to pass in Pandas DataFrames as inputs into a flow? When I do so I'm getting this error linked in the thread
Copy code
Traceback (most recent call last):
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/engine.py", line 2103, in orchestrate_task_run
    result = await call.aresult()
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/_internal/concurrency/calls.py", line 327, in aresult
    return await asyncio.wrap_future(self.future)
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/_internal/concurrency/calls.py", line 389, in _run_async
    result = await coro
  File "flows/orchestrator/orchestrator.py", line 191, in execute_node
    run_result = await flow_run
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/client/utilities.py", line 100, in with_injected_client
    return await fn(*args, **kwargs)
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/deployments/deployments.py", line 213, in run_deployment
    flow_run = await client.create_flow_run_from_deployment(
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/client/orchestration.py", line 625, in create_flow_run_from_deployment
    json=flow_run_create.dict(json_compatible=True, exclude_unset=True),
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/_internal/schemas/bases.py", line 141, in dict
    return json.loads(self.json(*args, **kwargs))
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/_internal/schemas/bases.py", line 111, in json
    return super().json(*args, **kwargs)
  File "pydantic/main.py", line 504, in pydantic.main.BaseModel.json
  File "/Users/shahm/Library/Caches/pypoetry/virtualenvs/ept-KCr_UkSx-py3.9/lib/python3.9/site-packages/prefect/_internal/schemas/serializers.py", line 24, in orjson_dumps_extra_compatible
    return orjson.dumps(
TypeError: Type is not JSON serializable: DataFrame
s
Hi @Moxit. Did you put DataFrame as Parameters? Its not good idea that pass too large Parameters like DataFrame. But you can try pass JSON parameters as dict or something can be JSON serialized
Copy code
@flow
def a(data: dict = {"A":[1,2,3],"B":[1,2,3]}):
    df = pd.DataFrame(data)