https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I started seeing a 422 error after up...
# ask-marvin
w
@Marvin I started seeing a 422 error after upgrading from Prefect 3.6.16 to 3.6.19
Copy code
PrefectHTTPStatusError('Client error \'422 Unprocessable Entity\' for url \'https://[domain]/api/task_runs/
{\'exception_message\': \'Invalid request received.\', \'exception_detail\': [{\'type\': \'model_attributes_type\', \'loc\': [\'body\'], \'msg\': \'Input should be a valid dictionary or object to extract fields from\', \'input\': \'{"state":{"type":"PENDING","name":"Pending","message":null,"state_details":{"flow_run_id":null,"task_run_id":null,"child_flow_run_id":null,"scheduled_time":null,"cache_key":null,"cache_expiration":null,"deferred":null,"untrackable_result":false,"pause_timeout":null,"pause_reschedule":false,"pause_key":null,"run_input_keyset":null,"refresh_cache":null,"retriable":null,"transition_id":null,"task_parameters_id":null,"traceparent":null,"deployment_concurrency_lease_id":null},"data":null},"name":"rss-ingestion-flow-0","flow_run_id":"019c9f49-8b4b-7889-ad2b-23ed9bf22816","task_key":"rss_ingestion_flow-ae4b41f6","dynamic_key":"0","cache_key":null,"cache_expiration":null,"task_version":"8a09eee71569be37fe5afcca9ea22e2d","empirical_policy":{"max_retries":0,"retry_delay_seconds":0.0,"retries":0,"retry_delay":0,"retry_jitter_factor":null},"tags":[],"labels":{},"task_inputs":{"integration":[],"cutoff_date_override":[]}}\'}], \'request_body\': \'{"state":{"type":"PENDING","name":"Pending","message":null,"state_details":{"flow_run_id":null,"task_run_id":null,"child_flow_run_id":null,"scheduled_time":null,"cache_key":null,"cache_expiration":null,"deferred":null,"untrackable_result":false,"pause_timeout":null,"pause_reschedule":false,"pause_key":null,"run_input_keyset":null,"refresh_cache":null,"retriable":null,"transition_id":null,"task_parameters_id":null,"traceparent":null,"deployment_concurrency_lease_id":null},"data":null},"name":"rss-ingestion-flow-0","flow_run_id":"019c9f49-8b4b-7889-ad2b-23ed9bf22816","task_key":"rss_ingestion_flow-ae4b41f6","dynamic_key":"0","cache_key":null,"cache_expiration":null,"task_version":"8a09eee71569be37fe5afcca9ea22e2d","empirical_policy":{"max_retries":0,"retry_delay_seconds":0.0,"retries":0,"retry_delay":0,"retry_jitter_factor":null},"tags":[],"labels":{},"task_inputs":{"integration":[],"cutoff_date_override":[]}}\'}
m
✅ thought for 461.1 seconds
w
@Marvin this issue seems to happen when prefect client is on 3.6.16 and server on 3.6.19
m
✅ thought for 153.8 seconds
Thanks for the datapoint. In Prefect 3.x, a 3.6.16 client should be able to talk to a 3.6.19 server — there aren’t any known compatibility breaks between those patch versions, and the client is designed to ignore new server fields. So the version skew alone likely isn’t the root cause. Could you share: - The exact error/logs you’re seeing (with 
PREFECT_LOGGING_LEVEL=DEBUG
and 
PREFECT_DEBUG_MODE=1
) - Outputs of 
prefect version
on both client and server (to confirm API version and env) - How you’re starting the worker and any work pool details Quick things to try while we investigate: - Upgrade the client to match the server to rule out a fixed bug: 
pip install -U prefect
(or your env manager of choice) - Double-check 
PREFECT_API_URL
on the client - If you see a websocket “unable to authenticate to events” warning: that’s typically non-fatal and shouldn’t prevent a worker from starting or polling, but the lines immediately before/after that warning are important to diagnose For reference, Prefect 3.x release notes: - 3.6.19 release notes - 3.6.16 release notes If you can paste the DEBUG logs around the failure, I can pinpoint whether it’s network/auth, worker->pool connectivity, or something else.
Thanks for the details — that error message is a strong signal you’ve hit the recent FastAPI 0.132.0 change. What’s happening - FastAPI 0.132.0 became stricter about request bodies. If the body is sent as a raw JSON string (or without the proper Content-Type), the server sees it as a string instead of a JSON object and returns 422 with: Input should be a valid dictionary or object to extract fields from - In Prefect 3.6.16 the client posted pre-serialized JSON via httpx’s 
content=
param. With FastAPI 0.132.0 this can trigger the 422 you’re seeing. - Prefect 3.6.19 includes a fix: it now uses httpx’s 
json=
with a Python dict, which sets 
Content-Type: application/json
and avoids this error. References - Fix PR: Fix FastAPI 0.132.0 compatibility - Matching issue/report (same error text): FastAPI 0.132.0 causes 422 for sub-flows - 3.6.19 release notes (mentions the fix): changelog Why you might still see it after upgrading to 3.6.19 - One or more running processes (workers/agents/flow-runtime) are still on an older Prefect version that uses 
content=
(they need a restart to pick up 3.6.19). - You’re self-hosting and the server/proxy is overriding or stripping 
Content-Type: application/json
. - Mixed versions between client and server environments. What I recommend 1) Restart all Prefect processes so they actually run 3.6.19: - Workers/agents - Any long-lived services or containers that execute your flows 2) If you’re self-hosting, ensure the server image is also on 3.6.19+. 3) As a temporary workaround (only if you cannot upgrade server immediately), pin FastAPI on the server to < 0.132.0 (e.g., 0.131.0). 4) If it still happens: - Confirm versions on both sides and share them: - Client/worker: 
prefect version
- Server: check the image tag or 
/api/health
- Let me know if you’re behind a reverse proxy/load balancer that could alter 
Content-Type
. If you can share whether you’re on Prefect Cloud or self-hosted, plus the versions of your server and workers and whether they’ve been restarted since the upgrade, I can help pinpoint the next step quickly.
108 Views
Open in Slack
PreviousNext
Powered by