https://prefect.io logo
Powered by
Title
j

John Ramirez

03/09/2020, 6:30 PM
Traceback (most recent call last):
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/prefect/utilities/serialization.py", line 186, in _validate_json
    json.dumps(value)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type datetime is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 176, in <module>
    main()
  File "main.py", line 163, in main
    labels=[args.env],
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/prefect/core/flow.py", line 1412, in register
    no_url=no_url,
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/prefect/client/client.py", line 616, in register
    serialized_flow = flow.serialize(build=build)  # type: Any
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/prefect/core/flow.py", line 1209, in serialize
    serialized = schema(exclude=["storage"]).dump(self)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/schema.py", line 556, in dump
    result = self._serialize(processed_obj, many=many)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/schema.py", line 520, in _serialize
    value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/fields.py", line 316, in serialize
    return self._serialize(value, attr, obj, **kwargs)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/prefect/utilities/serialization.py", line 220, in _serialize
    return super()._serialize(value, attr, obj, **kwargs)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/fields.py", line 571, in _serialize
    return schema.dump(nested_obj, many=many)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/schema.py", line 556, in dump
    result = self._serialize(processed_obj, many=many)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/schema.py", line 516, in _serialize
    for d in typing.cast(typing.Iterable[_T], obj)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/schema.py", line 516, in <listcomp>
    for d in typing.cast(typing.Iterable[_T], obj)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/schema.py", line 520, in _serialize
    value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/marshmallow/fields.py", line 316, in serialize
    return self._serialize(value, attr, obj, **kwargs)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/prefect/utilities/serialization.py", line 181, in _serialize
    self._validate_json(value)
  File "/Users/johnramirez/.local/share/virtualenvs/xignite-data-x4Gzfino/lib/python3.7/site-packages/prefect/utilities/serialization.py", line 188, in _validate_json
    raise ValidationError("Value is not JSON-compatible")
marshmallow.exceptions.ValidationError: Value is not JSON-compatible
c

Chris White

03/09/2020, 6:55 PM
Hi John, how can we help? The traceback seems clear to me --> you have a datetime object somewhere that you shouldn’t have one (maybe as a 
Parameter
default?). Running 
flow.serialize()
should reproduce the error in a more lightweight fashion
j

John Ramirez

03/09/2020, 7:16 PM
can you highlight the error that identified the datetime error
c

Chris White

03/09/2020, 7:24 PM
Yea it’s up top: TypeError: Object of type datetime is not JSON serializable
s

StefanKjartansson

03/10/2020, 7:58 PM
@Chris White Is it possible to override the JSON serialization function?
c

Chris White

03/10/2020, 8:19 PM
@StefanKjartansson in the case above, the error comes from creating the metadata payload for Prefect Cloud, so while it is technically possible to override the serialization logic (found here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/serialization/flow.py) you’ll need to work on making sure it’s compatible with how Cloud stores / interacts with the metadata
s

StefanKjartansson

03/10/2020, 8:42 PM
Right, so theoretically, if I’d want to serialize f.i. a pathlib.Path instance, I expect that Cloud would need to understand a “://path” prefix for that to work. https://github.com/PrefectHQ/prefect/blob/master/src/prefect/utilities/serialization.py#L400:L409
c

Chris White

03/10/2020, 8:44 PM
yea, or you’d just want to create your own serializer that converts the path instance -> JSON and then have an appropriate deserializer running that can deserialize the JSON to a pathlib instance again
Because Prefect Cloud will only ever store JSON information, so the only surface area users can change are how you choose to deserialize the JSON payloads into Python objects
2 Views
#prefect-communityJoin Slack
Powered by