Shivam Bhatia

    Shivam Bhatia

    2 months ago
    Hi, I am trying to figure out running flows on a docker container (prefect 2.0b8) After the flow run is complete I get an error and my agent stops running. (Error log in the thread) How can i fix this? Any help would be appreciated
    09:07:33.923 | INFO    | Task run 'sort-92900a21-0' - Finished in state Completed()
    09:07:35.216 | INFO    | Flow run 'shrewd-shark' - Finished in state Completed('All states completed.')
    09:07:35.899 | INFO    | prefect.flow_runner.docker - Flow run container 'shrewd-shark' has status 'removing'
    Traceback (most recent call last):
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/cli/_utilities.py", line 44, in wrapper
        return fn(*args, **kwargs)
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 193, in wrapper
        return run_async_in_new_loop(async_fn, *args, **kwargs)
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 140, in run_async_in_new_loop
        return anyio.run(partial(__fn, *args, **kwargs))
      File "/home/shivam/.local/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 70, in run
        return asynclib.run(func, *args, **backend_options)
      File "/home/shivam/.local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 292, in run
        return native_run(wrapper(), debug=debug)
      File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/home/shivam/.local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
        return await func(*args)
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/cli/agent.py", line 69, in start
        await anyio.sleep(PREFECT_AGENT_QUERY_INTERVAL.value())
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/agent.py", line 221, in __aexit__
        await self.shutdown(*exc_info)
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/agent.py", line 210, in shutdown
        await self.task_group.__aexit__(*exc_info)
      File "/home/shivam/.local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
        raise exceptions[0]
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/flow_runners/docker.py", line 122, in submit_flow_run
        return await run_sync_in_worker_thread(self._watch_container, container_id)
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 56, in run_sync_in_worker_thread
        return await anyio.to_thread.run_sync(call, cancellable=True)
      File "/home/shivam/.local/lib/python3.9/site-packages/anyio/to_thread.py", line 31, in run_sync
        return await get_asynclib().run_sync_in_worker_thread(
      File "/home/shivam/.local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
        return await future
      File "/home/shivam/.local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run
        result = context.run(func, *args)
      File "/home/shivam/.local/lib/python3.9/site-packages/prefect/flow_runners/docker.py", line 347, in _watch_container
        result = container.wait()
      File "/home/shivam/.local/lib/python3.9/site-packages/docker/models/containers.py", line 512, in wait
        return self.client.api.wait(self.id, **kwargs)
      File "/home/shivam/.local/lib/python3.9/site-packages/docker/utils/decorators.py", line 19, in wrapped
        return f(self, resource_id, *args, **kwargs)
      File "/home/shivam/.local/lib/python3.9/site-packages/docker/api/container.py", line 1304, in wait
        return self._result(res, True)
      File "/home/shivam/.local/lib/python3.9/site-packages/docker/api/client.py", line 274, in _result
        self._raise_for_status(response)
      File "/home/shivam/.local/lib/python3.9/site-packages/docker/api/client.py", line 270, in _raise_for_status
        raise create_api_error_from_http_exception(e)
      File "/home/shivam/.local/lib/python3.9/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
        raise cls(e, response=response, explanation=explanation)
    docker.errors.NotFound: 404 Client Error for <http+docker://localhost/v1.41/containers/9b7ea717b9138fda4299ff6c52ec10d921619e8b0ba11168d07781f17a40581d/wait>: Not Found ("No such container: 9b7ea717b9138fda4299ff6c52ec10d921619e8b0ba11168d07781f17a40581d")
    An exception occurred.
    Anna Geller

    Anna Geller

    2 months ago
    Can you share your Deployment?