@Marvin a flow uses a combination of basic python and tasks. it submits tasks to be executed asyncronously using results = [task.submit(a, *args) for a in ['a1', 'a2']]. Though the tasks are completing successfully, I am getting a serialization error for the flow, which has persist_results=True using Pickle serializer. The object that can't be serialized is as follows. Suggest ways to resolve this [Completed(message=None, type=COMPLETED, result=UnpersistedResult(type='unpersisted', artifact_type='result', artifact_description='Unpersisted result of type `list`')), Completed(message=None, type=COMPLETED, result=UnpersistedResult(type='unpersisted', artifact_type='result', artifact_description='Unpersisted result of type `list`')), Completed(message=None, type=COMPLETED, result=UnpersistedResult(type='unpersisted', artifact_type='result', artifact_description='Unpersisted result of type `bool`'))]

It seems like the root issue is related to the

UnpersistedResult

objects which the

PickleSerializer

cannot handle. Here are a few strategies you could take to resolve this serialization error: 1. Ensure Tasks Persist Results: Make sure each task involved in your flow explicitly specifies that its results should be persisted. You can use the

persist_result

parameter in the

@task

decorator. For example:

python
   from prefect import task

   @task(persist_result=True)
   def my_task(input_data):
       return input_data

2. Define Custom Serializers: If the default

PickleSerializer

is causing issues, you might want to define a custom serializer using

cloudpickle

or any other pickle library. For example:

python
   from prefect.serializers import PickleSerializer
   pickle_serializer = PickleSerializer(picklelib="cloudpickle")

3. Check Task Results and Serialization: Make sure that the objects being returned from the tasks are serializable. Sometimes operations or objects within tasks are returned in a form that’s not compatible with the serialization process. 4. Implement Custom Result Handlers: If necessary, you can implement custom handlers for dealing with serialization or results. Prefect allows you to create custom result types as well. 5. Use Literal Results if Simplicity is Required: If your results are simple and don’t necessarily need persisting, consider using

LiteralResult

which is lightweight. Here are some helpful links from the documentation to guide you further: - Persisting Results in Prefect - Pickle Serializer in Prefect Would you like more specific examples or deeper insights into any of these strategies?