Zach Schumacher
06/09/2021, 2:09 PMZach Schumacher
06/09/2021, 2:09 PMprefect.utilities.exceptions.ClientError: [{'path': ['create_flow_from_compressed_string'], 'message': '1 validation error for FlowSchema\nrun_config\n value is not a valid dict (type=type_error.dict)', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]
Kevin Kho
Zach Schumacher
06/09/2021, 2:13 PMZach Schumacher
06/09/2021, 2:14 PMclass KubernetesRunConfig(KubernetesRun):
"""A KubernetesRun class that gives some warnings for some common prefect config 'gotchas'"""
def __init__(self, *args, **kwargs):
if "image_pull_policy" not in kwargs:
# this is the k8s default anyway, but its nice to be explicit so you can see it in the UI
kwargs.update(image_pull_policy="IfNotPresent")
if "job_template" not in kwargs and "job_template_path" not in kwargs:
kwargs.update(job_template=_ETL_NODE_TEMPLATE)
if "labels" not in kwargs or not any(label.startswith("k8s-") for label in kwargs["labels"]):
warnings.warn(
"Registering config with no k8s agent label. "
"Your label must match the agent you want to run on or your flow will never start!"
)
if "image" not in kwargs:
warnings.warn(
"No image passed. Is this intentional? "
"If you don't pass an image the flow run will use the base prefect image."
)
super().__init__(*args, **kwargs)
Zach Schumacher
06/09/2021, 2:14 PMrun_config=KubernetesRunConfig(labels=[settings.PREFECT_AGENT_LABEL], image=settings.PREFECT_IMAGE),
Kevin Kho
ValueError: Flow could not be deserialized successfully. Error was: TypeError('not all arguments converted during string formatting')
. Will look into a bit more laterKevin Kho
Zach Schumacher
06/09/2021, 3:12 PMZach Schumacher
06/09/2021, 3:14 PMclass MyUberSchema(OneOfSchema):
type_schemas = {"foo": FooSchema, "bar": BarSchema}
def get_obj_type(self, obj):
if isinstance(obj, Foo):
return "foo"
elif isinstance(obj, Bar):
return "bar"
else:
raise Exception("Unknown object type: {}".format(obj.__class__.__name__))
Zach Schumacher
06/09/2021, 3:14 PMget_obj_type
funcZanie
Zanie
Zach Schumacher
06/09/2021, 3:17 PMZach Schumacher
06/09/2021, 3:17 PM