https://prefect.io logo
Join Slack
Powered by
Hey everyone, just updated to 2.4.2 and all my flo...
# prefect-community
g
Hey everyone, just updated to 2.4.2 and all my flows stop working, did I miss a braking change. The error is Response: {'detail': 'protected block types cannot be updated.'}. The strange thing is that the block that is accessed is of type local-file-system and I do not have such block in my system.
✅ 1
a
Thank you so much for reporting this, are you on Cloud? when is this error thrown? is it when you run any flow from deployment (i.e. due to storage and infra blocks)?
g
No it is dockersed local server and client
Loading flow for deployment 'ingest_images_flow'...
121434 AM DEBUG Starting 'DaskTaskRunner'; submitted tasks will be run concurrently... 121435 AM ERROR Crash detected! Execution was interrupted by an unexpected exception. 121443 AM DEBUG Crash details: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 1321, in report_flow_run_crashes yield File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 364, in begin_flow_run await result_filesystem._save(is_anonymous=True) File "/usr/local/lib/python3.8/site-packages/prefect/blocks/core.py", line 717, in _save await self.register_type_and_schema(client=client) File "/usr/local/lib/python3.8/site-packages/prefect/blocks/core.py", line 662, in register_type_and_schema await client.update_block_type( File "/usr/local/lib/python3.8/site-packages/prefect/client.py", line 1150, in update_block_type await self._client.patch( File "/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1916, in patch return await self.request( File "/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1527, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "/usr/local/lib/python3.8/site-packages/prefect/client.py", line 279, in send response.raise_for_status() File "/usr/local/lib/python3.8/site-packages/prefect/client.py", line 225, in raise_for_status raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.cause prefect.exceptions.PrefectHTTPStatusError: Client error '403 Forbidden' for url 'http://10.19.8.85:4210/api/block_types/a2dda662-16a7-4f86-ba75-e99a0192d5ff' Response: {'detail': 'protected block types cannot be updated.'} For more information check: https://httpstatuses.com/403
here is the complete log with a stack trace produced, after I run a deployment from the UI
a
rather than logs, it would be more helpful to first share how did you get there, how did you configure your agent, API server, which versions are there I believe some component in your setup might be still on the old Prefect version as this release should fix the protected block types issue https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md#release-242
g
let me double check are all docker images changed
a
generally, since I don't see any immediate breaking change that could cause that, perhaps you could at first (to avoid further issues in your flows) downgrade to a lower version and to help identify the root cause of why this upgrade didn't work, it might be best to create a GitHub issue so we could help troubleshoot on Monday
The strange thing is that the block that is accessed is of type local-file-system and I do not have such block in my system.
based on this message, one thing you could try (regardless of whether you decide to keep 2.4.2 or downgrade) is to delete the deployment(s) that failed and create new ones
(potentially also recreating blocks)
g
it was easy to recreate all blocks and deployments, but still the same, will downgrade for now
actually digging deeper, your suggestion that part of the system was running an old version was spot on
some of the flows are defined in software and use docker infrastructure too and there the image was not updated to use the 2.4.2 version
👍 1
thanks for pointing me in the right direction
👍 2
a
great work figuring that out!
👍 1
3 Views
Open in Slack
PreviousNext
Powered by