We are getting this error during

prefect agent start -p 'name'

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 260, in coroutine_wrapper
    return call()
           ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 245, in __call__
    return self.result()
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 173, in result
    return self.future.result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 218, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/cli/deployment.py", line 1143, in build
    deployment = await Deployment.build_from_flow(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/deployments.py", line 783, in build_from_flow
    await deployment.load()
  File "/usr/local/lib/python3.11/site-packages/prefect/deployments.py", line 559, in load
    deployment = await client.read_deployment_by_name(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/client/orchestration.py", line 1513, in read_deployment_by_name
    return schemas.responses.DeploymentResponse.parse_obj(response.json())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pydantic/main.py", line 526, in pydantic.main.BaseModel.parse_obj
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 8 validation errors for DeploymentResponse
schedule -> interval
  field required (type=value_error.missing)
schedule -> timezone
  argument of type 'function' is not iterable (type=type_error)
schedule -> cron
  extra fields not permitted (type=value_error.extra)
schedule -> day_or
  extra fields not permitted (type=value_error.extra)
schedule -> timezone
  argument of type 'function' is not iterable (type=type_error)
schedule -> rrule
  field required (type=value_error.missing)
schedule -> cron
  extra fields not permitted (type=value_error.extra)
schedule -> day_or
  extra fields not permitted (type=value_error.extra)
An exception occurred.

Is anybody having this issue.

hi @nicholasnet - yes, we're investigating this issue now. can you share your

prefect version

? it should not be a problem on newer versions of prefect. if possible, could you also share your pydantic version?

Prefect version 2.10.8 Pydantic >= 1.10.11, < 2.0.0

🧐

Thank you very much for looking into this especially considering month and time.

would you be able to share

prefect deployment inspect <your deployment>

for the one its failing to read there?

Since its failing before creating the YAML I am not sure I can inspect anything. It is failing right here

prefect deployments build flow/main.py:nightly_job --name $PROJECT-$BRANCH_NAME-nightly-job --infra-block kubernetes-job/$PROJECT-$BRANCH_NAME --pool '***-'$ENV --work-queue 'default' --tag 'nightly' --apply

gotcha, i was thinking there might be an existing deployment that you were trying to run with your agent since you said

We are getting this error during

prefect agent start -p 'name'

Same command is working for other deployment though. For instance this one

prefect deployments build flow/transform.py:transform_data --name $PROJECT-$BRANCH_NAME-transform --infra-block kubernetes-job/$PROJECT-$BRANCH_NAME --pool 'bidw-'$ENV --work-queue 'default' --tag 'nightly' --apply

and even previous one is working but in another branch (feature branch)

in the feature branch, do you have the same deps?

Yeah same deps

ok sorry one more question if you dont mind, would you be able to share the output of

prefect deployment inspect <your deployment>

from where it works in the feature branch?

I am trying

cat

to see if I get that yaml ...running the build now.

to clarify, I'm interested in the server representation of the deployment, so wherever you said it was working, you should be able to do

prefect deployment ls

and take the name and throw it into

prefect deployment inspect <the name>

- not so much the

deployment.yaml

I am running job in Kube cluster may be I can shell in to agent and try it?

you shouldnt need to exec in, you just need to ask the API for the deployment spec, which if your local machine is connected to your workspace, you should just be able to do the CLI commands i mentioned above

too late I was already in 🙂 anyway here is the inspect result

{
    'id': '1e3efbc0-ea9a-4ad2-9659-4e7dc0dad160',
    'created': '2023-08-02T12:45:50.422509+00:00',
    'updated': '2023-12-15T23:45:09.364869+00:00',
    'name': 'bidw-prefect-dev-nightly-job',
    'version': 'da80f14ade0e799151f3e7e43717c42b',
    'description': None,
    'flow_id': 'dc75465d-b81a-4924-9fd6-0d3c091e6d88',
    'schedule': {'cron': '30 21 * * 0-5', 'timezone': 'America/Denver', 'day_or': True},
    'is_schedule_active': True,
    'infra_overrides': {},
    'parameters': {},
    'tags': ['nightly'],
    'work_queue_name': 'default',
    'parameter_openapi_schema': {'type': 'object', 'title': 'Parameters', 'properties': {}},
    'path': '/opt/prefect/flows',
    'entrypoint': 'flow/main.py:nightly_job',
    'manifest_path': None,
    'storage_document_id': None,
    'infrastructure_document_id': '00d8c8e3-6eaa-4f35-aca7-2cc840e536ed',
    'created_by': {'id': 'f338de15-490c-4b81-833e-c075c907f8c3', 'type': 'BOT', 'display_value': 'non-prod-deployer'},
    'updated_by': {'id': 'f338de15-490c-4b81-833e-c075c907f8c3', 'type': 'BOT', 'display_value': 'non-prod-deployer'},
    'work_pool_name': 'bidw-dev',
    'infrastructure': {
        'type': 'kubernetes-job',
        'env': {'ENV': 'dev', 'BRANCH_NAME': 'dev', 'ENVIRONMENT': 'dev'},
        'labels': {},
        'name': None,
        'command': None,
        'image': 'xxxxxx.dkr.ecr.us-west-2.amazonaws.com/bidw-prefect:dev-82f5b6008f31fc417a826c2ae34b79e3f11db735',
        'namespace': 'dev-bidw',
        'service_account_name': None,
        'image_pull_policy': <KubernetesImagePullPolicy.ALWAYS: 'Always'>,
        'cluster_config': None,
        'job': {
            'kind': 'Job',
            'spec': {
                'template': {
                    'spec': {
                        'volumes': [{'name': 'bidw-volume', 'persistentVolumeClaim': {'claimName': 'px-prefect-pvc'}}],
                        'containers': [
                            {
                                'env': [{'name': 'PREFECT_LOCAL_STORAGE_PATH', 'value': '/mnt/data'}],
                                'name': 'prefect-job',
                                'envFrom': [{'secretRef': {'name': 'bidw-cloud'}}],
                                'volumeMounts': [{'name': 'bidw-volume', 'mountPath': '/mnt/data'}]
                            }
                        ],
                        'completions': 1,
                        'parallelism': 1,
                        'restartPolicy': 'Never'
                    }
                }
            },
            'metadata': {'labels': {}},
            'apiVersion': 'batch/v1'
        },
        'customizations': [],
        'job_watch_timeout_seconds': 6000,
        'pod_watch_timeout_seconds': 6000,
        'stream_output': True,
        'finished_job_ttl': 120,
        'block_type_slug': 'kubernetes-job'
    }
}

interesting, the second does not have a schedule I notice

yeah for feature we do not schedule only dev (main)

ok, i suspect that somehow the problem is related to the schedule object (cron specifically maybe?) - thank you for the info! I will let you know if I find a resolution

Sure. Thank you very much. I will debug on my side as well.

thank you @nicholasnet - we are actively looking into this!

By the way I also sent email to support regarding this yesterday. In case you want to collaborate.

thank you for the information

Sure no problem thank you very much for looking into this. If you need anymore information from my end please let me know.

hey! could you try pinning this dependency?

pendulum >= 2.1.2, < 3.0.0

Sure I will try it out