< Marvin> How can i get around this error when running `pref Prefect Community #ask-marvin

<@ULVA73B9P> How can i get around this error when ...

Pooyan

11/17/2023, 10:04 PM

@Marvin How can i get around this error when running

prefect deploy

Copy code

TypeError("Object of type 'FieldInfo' is not JSON serializable")

My parameters field is like

Copy code

parameters:
  training:
    key_1: value_1
    key_2: value_2

Nate

11/17/2023, 10:20 PM

do you have more of this trace?

Pooyan

11/17/2023, 10:26 PM

Not sure if it is related now. I made the parameters simple but still the same issue

Copy code

File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 168, in result
    return self.__get_result()
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/_internal/concurrency/calls.py", line 345, in _run_async
    result = await coro
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/cli/deploy.py", line 292, in deploy
    await _run_single_deploy(
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/cli/deploy.py", line 368, in _run_single_deploy
    flow = await register_flow(deploy_config["entrypoint"])
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/deployments/base.py", line 310, in register_flow
    flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, entrypoint)
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/flows.py", line 1498, in load_flow_from_entrypoint
    flow = import_object(entrypoint)
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/utilities/importtools.py", line 201, in import_object
    module = load_script_as_module(script_path)
  File "/Users/pooyan/miniconda3/envs/orchestrator/lib/python3.8/site-packages/prefect/utilities/importtools.py", line 164, in load_script_as_module
    raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'prefect-training-workflow.py' encountered an exception: TypeError("Object of type 'FieldInfo' is not JSON serializable")
An exception occurred.

Pooyan

11/17/2023, 10:52 PM

This is happening because of this import in my flow script:

Copy code

from prefect_spark_on_k8s_operator import (
   SparkApplication,
   run_spark_application
)

Pooyan

11/17/2023, 10:55 PM

Dockerfile:

Copy code

FROM prefecthq/prefect:2.14.3-python3.8
RUN pip install --upgrade pip
RUN pip install pydantic==1.10.13 prefect==2.13.0 prefect_kubernetes==0.3.0 \
    prefect_spark_on_k8s_operator==0.1.2 ruamel.yaml==0.18.5
COPY . /opt/prefect/training/
WORKDIR /opt/prefect/training/

Pooyan

11/17/2023, 10:59 PM

I think it is resolved once I updated pydantic version to 1.10.13 in my env

3 Views

Open in Slack

Previous Next