Blake Stefansen
11/08/2022, 7:27 PMOrion logging error
with 2.6.6 for our k8 jobs
Will post logs in the thread--- Orion logging error ---
The log worker encountered a fatal error.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/prefect/logging/handlers.py", line 82, in _send_logs_loop
anyio.run(self.send_logs)
File "/usr/local/lib/python3.10/dist-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 292, in run
return native_run(wrapper(), debug=debug)
File "/usr/lib/python3.10/asyncio/runners.py", line 47, in run
_cancel_all_tasks(loop)
File "/usr/lib/python3.10/asyncio/runners.py", line 56, in _cancel_all_tasks
to_cancel = tasks.all_tasks(loop)
File "/usr/lib/python3.10/asyncio/tasks.py", line 54, in all_tasks
tasks = list(_all_tasks)
File "/usr/lib/python3.10/_weakrefset.py", line 65, in __iter__
for itemref in self.data:
RuntimeError: Set changed size during iteration
Worker information:
Approximate queue length: 0
Pending log batch length: 0
Pending log batch size: 0
from prefect.deployments import run_deployment
run_deployment(
name='geocoder-flow/geocoder_yaml_deploy_k8',
parameters=geocoder_params,
flow_run_name=flow_run_name,
timeout=0
)
Mason Menges
11/11/2022, 10:31 PM