<@ULVA73B9P> when passing an object between tasks ...
# ask-marvin
n
@Marvin when passing an object between tasks (without results caching) is the python object serialized/deserialized in the process?
m
thought for 64.8 seconds
Yes, when passing objects between tasks in Prefect, the data is always serialized and deserialized, regardless of whether results caching is enabled. This is necessary because tasks may run in different processes or even on different machines. By default, Prefect uses a combination of JSON and Pickle serializers to handle common Python objects. The serialization process converts Python objects into a byte format that can be safely transmitted between tasks, and then deserializes them back into Python objects when needed by the receiving task. Here's what you should know: 1. This happens automatically - you don't need to handle serialization manually 2. Most common Python objects (lists, dictionaries, numbers, strings, etc.) will work without any special handling 3. For complex objects that aren't natively serializable, you might need to implement custom serialization logic using Pydantic models If you're working with objects that are giving you serialization troubles, you can find more details about custom serialization in the serializers documentation. Would you like me to elaborate on any particular aspect of the serialization process, such as handling custom objects or configuring different serializers?
k
uh.. no
n
is Marvin offbase on this? -- so it should pass the python object between tasks without serialization/deserialization if the tasks are on the same instance?
k
I believe so. there's no reason for serialization to happen if the result isn't being written somewhere
👍 1
🙏 1