result = pickle.dumps(x, **dump_kwargs)
_pickle.PicklingError: Can't pickle <cyfunction xxxx at 0x7fff2e711390>: it's not the same object as pipeline.xxxx
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/engine/flow_runner.py", line 628, in get_flow_run_state
extra_context=extra_context(task),
File "/usr/local/lib/python3.7/site-packages/prefect/executors/dask.py", line 422, in submit
_ma
✅ 1
b
Bianca Hoch
09/13/2022, 9:28 PM
Hey John, what version of Prefect are you using? Can you also provide a minimal reproducible example of your code?
j
John Song
09/13/2022, 10:17 PM
FROM prefecthq/prefect:latest
I am using it in docker container, I am pulling from the latest
let me create a sample code to reproduce this problem
@Bianca Hoch I just attached a simple sample to reproduce it, the sample file is under test folder. I also included setup.py and script to cythonize it. There is run.sh to run the cythonized code. you will see the error
I actually resolved the issue if I don't use inline task
✨ 1
👍 1
🚀 1
k
Khuyen Tran
09/14/2022, 3:25 PM
Do you mind condensing your messages into one thread? It would make it easier for others to follow
j
John Song
09/15/2022, 5:14 PM
In the simplest term, @task won't work with cythonized code