https://prefect.io logo
#prefect-community
Title
# prefect-community
n

Naga Sravika Bodapati

05/13/2022, 2:00 PM
Seeing this error: can't pickle CompiledFFI objects , can someone help what might be the issue?
1
a

Anna Geller

05/13/2022, 2:03 PM
you would need to provide a bit more information 🙂
is it for Prefect 1.0 or 2.0?
usually, pickle errors occur due to flow serialization issues, often happens when you serialized your flow with one Python and Prefect version during registration, but then ran it in an environment that had different versions
n

Naga Sravika Bodapati

05/13/2022, 2:06 PM
prefect 1.0
I am trying to register the flow
and seeing this error
in prefect 1.0, is there a way to call one task from another task function?
k

Kevin Kho

05/13/2022, 2:09 PM
You need to call
mytask.run()
but that’s calling the Python function under the hood. It’s not a task with results and retries etc
a

Anna Geller

05/13/2022, 2:09 PM
theoretically yes, you can invoke the .run() method but it's not recommended
n

Naga Sravika Bodapati

05/13/2022, 2:10 PM
ok
found the issue - i am using storage client for bigquery in one of the tasks and that is throwing this error. Why is it happening?
can we remove this pickling?
k

Kevin Kho

05/13/2022, 2:40 PM
What storage are you using? You can store it as a script, but a client instantiated inside a task should be fine for registration because that code is not actually run yet
n

Naga Sravika Bodapati

05/13/2022, 2:41 PM
yes, i am not sure why i am seeing this because i have used similar logic elsewhere and it worked
GCS bucket
k

Kevin Kho

05/13/2022, 2:44 PM
You can use the
stored_as_script
=
True
to not have serialization during the registration time
You can find more info the in docstrings here
n

Naga Sravika Bodapati

05/13/2022, 2:46 PM
i tried to run the script on a different prefect vm
and registration went through!
what must have changed?
this first vm is on GCP and the second is on premise - will it make a difference?
k

Kevin Kho

05/13/2022, 2:56 PM
No that shouldn’t 😅.
n

Naga Sravika Bodapati

05/13/2022, 2:57 PM
yes! I am not understanding this! Prefect version on both is the same.
5 Views