Khyaati Jindal
01/06/2023, 11:27 AM"[Errno 28] No space left on device:
got this error during the agent run.James Zhang
01/06/2023, 12:54 PMprefect.deployments.run_deployment()
, is it possible to start these deployment runs concurrently?chara
01/06/2023, 2:00 PMDavid Cheedie
01/06/2023, 2:20 PMhttpx.HTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/api/accounts/8ff05591-b922-43d6-96e2-0a956c1a55e7/workspaces/c573eb73-711f-42a5-a82d-f11e6defcd82/flow_runs/2c132385-8d43-471c-9dc5-dfad65eb5e41/set_state>'
Encountered exception during execution:
Traceback (most recent call last):
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/prefect/client/orion.py", line 1564, in set_flow_run_state
response = await <http://self._client.post|self._client.post>(
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post
return await self.request(
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/prefect/client/base.py", line 251, in send
response.raise_for_status()
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/httpx/_models.py", line 745, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/api/accounts/8ff05591-b922-43d6-96e2-0a956c1a55e7/workspaces/c573eb73-711f-42a5-a82d-f11e6defcd82/flow_runs/2c132385-8d43-471c-9dc5-dfad65eb5e41/set_state>'
For more information check: <https://httpstatuses.com/404>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/prefect/engine.py", line 1574, in report_flow_run_crashes
yield
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/prefect/engine.py", line 514, in create_and_begin_subflow_run
terminal_state = await orchestrate_flow_run(
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/prefect/engine.py", line 599, in orchestrate_flow_run
state = await propose_state(client, Running(), flow_run_id=flow_run.id)
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/prefect/engine.py", line 1741, in propose_state
response = await client.set_flow_run_state(
File "/home/analytics/miniconda3/envs/prefect_env/lib/python3.10/site-packages/prefect/client/orion.py", line 1570, in set_flow_run_state
raise prefect.exceptions.ObjectNotFound(http_exc=e) from e
prefect.exceptions.ObjectNotFound
Tadej Svetina
01/06/2023, 2:41 PMJelle Vegter
01/06/2023, 2:27 PMNathan Low
01/06/2023, 4:56 PMalex
01/06/2023, 5:14 PMArun Giridharan
01/06/2023, 6:02 PMIlya Galperin
01/06/2023, 6:53 PMprefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
(full traceback in thread)Billy McMonagle
01/06/2023, 9:19 PMGuy Altman
01/06/2023, 9:23 PMSlackbot
01/06/2023, 9:44 PMjack
01/06/2023, 10:34 PMSiva Balusu
01/07/2023, 3:16 AMFlow could not be retrieved from deployment.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 260, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/prefect/deployments.py", line 167, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 463, in get_directory
return await self.filesystem.get_directory(
File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 308, in get_directory
return self.filesystem.get(from_path, local_path, recursive=True)
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 113, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 98, in sync
raise return_result
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 53, in _runner
result[0] = await coro
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 551, in _get
rpaths = await self._expand_path(rpath, recursive=recursive)
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 751, in _expand_path
out = await self._expand_path([path], recursive, maxdepth)
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 775, in _expand_path
raise FileNotFoundError(path)
FileNotFoundError: ['prefect-bucket/prefect_test2']
[7:10 PM] @here any help is much appreciated
[7:10 PM] agent is in kubernetes
[7:10 PM] I have no name!@prefect-agent-656bfcdc7d-pqjzx:~$ pip list | grep "prefect"
prefect 2.7.1
prefect-aws 0.2.3
prefect-dask 0.2.2
prefect-kv 0.1.0
prefect-snowflake 0.25
Khyaati Jindal
01/07/2023, 5:41 AM20:01:27.858 | INFO | Flow run 'lovely-emu' - Created task run 'RAREA12to13-2f7bbdf5-0' for task 'RAREA12to13'
20:01:27.864 | INFO | Flow run 'lovely-emu' - Executing 'RAREA12to13-2f7bbdf5-0' immediately...
20:01:29.858 | INFO | Task run 'RAREA12to13-2f7bbdf5-0' - Finished in state Completed()
20:01:30.235 | INFO | Flow run 'lovely-emu' - Finished in state Completed('All states completed.')
05:28:05.934 | INFO | prefect.infrastructure.process - Opening process 'augmented-warthog'...
05:28:16.139 | INFO | prefect.infrastructure.process - Process 'quaint-barracuda' exited cleanly.
05:28:25.094 | INFO | prefect.infrastructure.process - Process 'benign-myna' exited cleanly.
05:28:35.179 | INFO | prefect.infrastructure.process - Process 'pastel-swine' exited cleanly.
05:28:35.180 | INFO | prefect.infrastructure.process - Opening process 'helpful-turkey'...
05:28:35.230 | INFO | prefect.infrastructure.process - Opening process 'enormous-gecko'...
05:28:55.267 | INFO | prefect.infrastructure.process - Process 'lumpy-wallaby' exited cleanly. 05:29:13.009 | INFO | prefect.infrastructure.process - Process 'rigorous-elephant' exited cleanly.
05:29:31.230 | INFO | prefect.infrastructure.process - Process 'lovely-emu' exited cleanly.20:01:27.858 | INFO | Flow run 'lovely-emu' - Created task run 'RAREA12to13-2f7bbdf5-0' for task 'RAREA12to13'
20:01:27.864 | INFO | Flow run 'lovely-emu' - Executing 'RAREA12to13-2f7bbdf5-0' immediately...
20:01:29.858 | INFO | Task run 'RAREA12to13-2f7bbdf5-0' - Finished in state Completed()
20:01:30.235 | INFO | Flow run 'lovely-emu' - Finished in state Completed('All states completed.')
05:28:05.934 | INFO | prefect.infrastructure.process - Opening process 'augmented-warthog'...
05:28:16.139 | INFO | prefect.infrastructure.process - Process 'quaint-barracuda' exited cleanly.
05:28:25.094 | INFO | prefect.infrastructure.process - Process 'benign-myna' exited cleanly.
05:28:35.179 | INFO | prefect.infrastructure.process - Process 'pastel-swine' exited cleanly.
05:28:35.180 | INFO | prefect.infrastructure.process - Opening process 'helpful-turkey'...
05:28:35.230 | INFO | prefect.infrastructure.process - Opening process 'enormous-gecko'...
05:28:55.267 | INFO | prefect.infrastructure.process - Process 'lumpy-wallaby' exited cleanly. 05:29:13.009 | INFO | prefect.infrastructure.process - Process 'rigorous-elephant' exited cleanly.
05:29:31.230 | INFO | prefect.infrastructure.process - Process 'lovely-emu' exited cleanly.
Another weird thing about this is , even though I have enabled notifications for these flows from the prefect UI, no notifications were sent for the flow being in 'late' state.Stephen Herron
01/07/2023, 5:00 PMprefect-snowflake
YD
01/07/2023, 8:57 PM$ prefect version
Version: 2.7.7
API version: 0.8.4
Python version: 3.8.10
Git commit: e8ca30b8
Built: Fri, Jan 6, 2023 4:25 PM
OS/Arch: linux/aarch64
Profile: default
Server type: cloud
Error:
$ prefect deployment apply run_collect_stocks_data-deployment.yaml
Successfully loaded 'yquark_deployment'
Traceback (most recent call last):
File "/home/youval/.local/lib/python3.8/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
return fn(*args, **kwargs)
File "/home/youval/.local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 230, in coroutine_wrapper
return run_async_in_new_loop(async_fn, *args, **kwargs)
File "/home/youval/.local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 181, in run_async_in_new_loop
return anyio.run(partial(__fn, *args, **kwargs))
File "/home/youval/.local/lib/python3.8/site-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/home/youval/.local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 292, in run
return native_run(wrapper(), debug=debug)
File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/home/youval/.local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "/home/youval/.local/lib/python3.8/site-packages/prefect/cli/deployment.py", line 594, in apply
deployment_id = await deployment.apply()
File "/home/youval/.local/lib/python3.8/site-packages/prefect/deployments.py", line 633, in apply
deployment_id = await client.create_deployment(
File "/home/youval/.local/lib/python3.8/site-packages/prefect/client/orion.py", line 1313, in create_deployment
response = await <http://self._client.post|self._client.post>(
File "/home/youval/.local/lib/python3.8/site-packages/httpx/_client.py", line 1848, in post
return await self.request(
File "/home/youval/.local/lib/python3.8/site-packages/httpx/_client.py", line 1533, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/home/youval/.local/lib/python3.8/site-packages/prefect/client/base.py", line 251, in send
response.raise_for_status()
File "/home/youval/.local/lib/python3.8/site-packages/httpx/_models.py", line 745, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '422 Unprocessable Entity' for url '<https://api.prefect.cloud/api/accounts>
YD
01/08/2023, 2:29 AMnumpy 1.24.1
and I get an error when running:
$ prefect deployment build ...
the error is:
encountered an exception: AttributeError("module 'numpy' has no attribute 'typeDict'")
Andy Yeung
01/08/2023, 6:15 AMYaron Levi
01/08/2023, 1:00 PMTadej Svetina
01/08/2023, 8:11 PMTadej Svetina
01/08/2023, 9:28 PMasyncio.sleep
to wait between querying the API. Running of this task occupies 100% of the CPU - how can I fix this?Ben Muller
01/08/2023, 10:50 PMprefect deployment apply my-deployment.yml, mydeployment2.yml --upload
Error in 🧵Dylan Lim
01/09/2023, 1:44 AMDylan Lim
01/09/2023, 1:53 AMPostgresFetch
, transforms it and loads it onto our warehouse using PostgresExecuteMany
. The Flow gets stuck at the stage where it reads from the external db and when I cancel it via the UI, it also cancels indefinitely (I have flows that have been ‘cancelling’ since a few days ago). Can anyone point me towards any suggested areas to investigate? For context, I checked if it’s a volume issue but the number of rows historically retrieved are roughly the same amount, and other Flows that query from the same external db are not running into this issue.lialzm
01/09/2023, 8:01 AMKelvin DeCosta
01/09/2023, 9:27 AMDanilo Drobac
01/09/2023, 10:39 AMMax Rostron
01/09/2023, 11:01 AM