Pedro Machado
06/11/2021, 1:36 PMPrefectResult
in a flow that has a resource manager. The idea is to easily allow flow restarts without having to set up a different result backend.
However, I am running into an issue with the resource manager tasks. The setup
method of the resource manager returns an object that is not json serializable which causes the task to fail. Do you have any suggestions to workaround this? Is the only solution to use a different result backend for this task? Thanks!Kevin Kho
class NoOpSerializer(Serializer):
"""A `Serializer` that does nothing."""
def serialize(self, value):
return value
def deserialize(self, value):
return value
Kevin Kho
PrefectResult(serializer=NoOpSerializer())
Pedro Machado
06/11/2021, 1:42 PMPedro Machado
06/11/2021, 1:52 PMTypeError: PrefectResult only supports JSONSerializer or DateTimeSerializer
I am going to try to use a different result backend for this task (I still have PrefectResult
at the flow level).
Currently, my resource manager is defined in a different python module as follows:
@resource_manager
class TableauServer:
...
then, it's imported in the flow and used as a context manager:
with TableauServer(
user=tableau_user,
password=tableau_password,
server_url=tableau_server_url,
site=tableau_site,
) as server:
...
I tried assigning a result
to the context manager and it did not work. Do you have any suggestions to allow me to define the resource manager in a separate module and use a results class when using it?Kevin Kho
with Flow("example") as flow:
with MyResource(...) as resource:
some_task(resource)
other_task(resource)
Kevin Kho
some_task
and other_task
Pedro Machado
06/11/2021, 2:17 PMsetup
task from the flow. The resource manager is not essential in this case, but it would be nice to figure out a pattern to be able to use PrefectResults at the flow level with resource managers when the resource is not json serializable.Pedro Machado
06/11/2021, 2:18 PMKevin Kho