https://prefect.io
Join Slack
result = pickle.dumps(x, **dump_kwargs) _pickle.PicklingError: Can't pickle <cyfunction xxxx at 0...
# ask-community
j
result = pickle.dumps(x, **dump_kwargs) _pickle.PicklingError: Can't pickle <cyfunction xxxx at 0x7fff2e711390>: it's not the same object as pipeline.xxxx During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner new_state = method(self, state, *args, **kwargs) File "/usr/local/lib/python3.7/site-packages/prefect/engine/flow_runner.py", line 628, in get_flow_run_state extra_context=extra_context(task), File "/usr/local/lib/python3.7/site-packages/prefect/executors/dask.py", line 422, in submit _ma
✅ 1
b
Hey John, what version of Prefect are you using? Can you also provide a minimal reproducible example of your code?
j
FROM prefecthq/prefect:latest
I am using it in docker container, I am pulling from the latest
let me create a sample code to reproduce this problem
@Bianca Hoch I just attached a simple sample to reproduce it, the sample file is under test folder. I also included setup.py and script to cythonize it. There is run.sh to run the cythonized code. you will see the error
I actually resolved the issue if I don't use inline task
✨ 1
👍 1
🚀 1
k
Do you mind condensing your messages into one thread? It would make it easier for others to follow
j
In the simplest term, @task won't work with cythonized code
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by