Hi all I deployed a Prefect 3 1 1 flow successfully on ECS T Prefect Community #ask-community

Hi all, I deployed a Prefect (3.1.1) flow successf...

Moe

02/05/2025, 4:52 PM

Hi all, I deployed a Prefect (3.1.1) flow successfully on ECS. Then I added Cron schedules for it through the UI. But in Prefect it is crashing. Looking at AWS Cloudwatch logs this is what I see:

Copy code

2025-02-05T16:36:06.234Z
pydantic_core._pydantic_core.ValidationError: 10 validation errors for DeploymentResponse
2025-02-05T16:36:06.234Z
schedules.0.schedule.function-after[validate_timezone(), IntervalSchedule].interval
2025-02-05T16:36:06.234Z
Field required [type=missing, input_value={'cron': '35 14 * * 7', '...oronto', 'day_or': True}, input_type=dict]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/missing>
2025-02-05T16:36:06.234Z
schedules.0.schedule.function-after[validate_timezone(), IntervalSchedule].cron
2025-02-05T16:36:06.234Z
Extra inputs are not permitted [type=extra_forbidden, input_value='35 14 * * 7', input_type=str]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/extra_forbidden>
2025-02-05T16:36:06.234Z
schedules.0.schedule.function-after[validate_timezone(), IntervalSchedule].day_or
2025-02-05T16:36:06.234Z
Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/extra_forbidden>
2025-02-05T16:36:06.234Z
schedules.0.schedule.CronSchedule.cron
2025-02-05T16:36:06.234Z
Value error, Invalid cron string: "35 14 * * 7" [type=value_error, input_value='35 14 * * 7', input_type=str]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/value_error>
2025-02-05T16:36:06.234Z
schedules.0.schedule.RRuleSchedule.rrule
2025-02-05T16:36:06.234Z
Field required [type=missing, input_value={'cron': '35 14 * * 7', '...oronto', 'day_or': True}, input_type=dict]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/missing>
2025-02-05T16:36:06.234Z
schedules.0.schedule.RRuleSchedule.cron
2025-02-05T16:36:06.234Z
Extra inputs are not permitted [type=extra_forbidden, input_value='35 14 * * 7', input_type=str]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/extra_forbidden>
2025-02-05T16:36:06.234Z
schedules.0.schedule.RRuleSchedule.day_or
2025-02-05T16:36:06.234Z
Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/extra_forbidden>
2025-02-05T16:36:06.234Z
schedules.0.schedule.NoSchedule.cron
2025-02-05T16:36:06.234Z
Extra inputs are not permitted [type=extra_forbidden, input_value='35 14 * * 7', input_type=str]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/extra_forbidden>
2025-02-05T16:36:06.234Z
schedules.0.schedule.NoSchedule.timezone
2025-02-05T16:36:06.234Z
Extra inputs are not permitted [type=extra_forbidden, input_value='America/Toronto', input_type=str]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/extra_forbidden>
2025-02-05T16:36:06.234Z
schedules.0.schedule.NoSchedule.day_or
2025-02-05T16:36:06.234Z
Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool]
2025-02-05T16:36:06.234Z
For further information visit <https://errors.pydantic.dev/2.9/v/extra_forbidden>

I haven't seen this issue before, and nothing wrong with my schedule from what I can tell. What's happening here? These are my schedules I set in UI:

Nate

02/05/2025, 4:58 PM

hi @Moe - that's odd. these are worker logs you're showing? can you confirm whether you're using cloud or oss and the versions of your prefect client / worker?

Moe

02/05/2025, 5:01 PM

I'm using Prefect Cloud

Moe

02/05/2025, 5:05 PM

When I removed one of the schedules, it ran fine.

Nate

02/05/2025, 5:16 PM

I'm using Prefect Cloud

thanks! can you share the versions i see you said

Hi all, I deployed a Prefect (3.1.1) flow successfully on ECS

but from this its unclear if you mean the runtime image or the worker

Moe

02/05/2025, 5:57 PM

Prefect version:

Moe

02/05/2025, 5:59 PM

image tag:

prefecthq/prefect:3.1.1-python3.12

Nate

02/05/2025, 6:13 PM

what about your worker? i'd guess that's running as an ECS service?

Moe

02/05/2025, 6:15 PM

We're using ECS push workpool w/ Fargate. So I believe worker is handled by Prefect internally.

Nate

02/05/2025, 6:16 PM

gotcha gotcha, yep that sounds right

Nate

02/05/2025, 6:16 PM

hmm. have you tried upgrading the image tag? i don't specifically remember any issues with 3.1.1 though... it looks like some sort of schema mismatch

Moe

02/05/2025, 6:35 PM

@Nate I figured it out. Whenever I have a cron schedule set using Sunday as 7 e.g.

0 1 * * 7

, Prefect doesn't like it. But if I used

0 1 * * 0

, flow runs fine. And it fails even schedule is paused.

Moe

02/05/2025, 6:35 PM

Although UI seems to recognize it as Sunday for 7 still.

Moe

02/05/2025, 6:48 PM

Also realizing this issue does not occur in our non-ECS workpool i.e. works fine on a workpool set up on EC2 instance.

8 Views

Open in Slack

Previous Next