https://prefect.io logo
Join Slack
Powered by
I’m getting the following error when starting the ...
# pacc-june-14-2023
j
I’m getting the following error when starting the worker. What could be the issue?
j
Hey Jeff, if you stop and rerun it should work. Let me know if it does. We will automatically retry these so it shouldn’t be an issue after tomorrow’s 2.10.14 release.
If you want to move your traceback to this thread that might help keep the main channel a little more tidy. 🙂
j
Thanks Jeff. Let me give that a try
👍 1
Copy code
$ prefect worker start --pool 'workerpool1'
Discovered worker type 'process' for work pool 'workerpool1'.
Worker 'ProcessWorker 554a8335-b13c-43c1-9b86-5055990e4226' started!
21:16:16.088 | ERROR   | prefect.server - Encountered exception in request:
Traceback (most recent call last):
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
    self.dialect.do_execute(
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
    cursor.execute(statement, parameters)
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 113, in execute
    self._adapt_connection._handle_exception(error)
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 263, in _handle_exception
    raise error
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 95, in execute
    self.await_(_cursor.execute(operation, parameters))
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 125, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 185, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/aiosqlite/cursor.py", line 48, in execute
    await self._execute(self._cursor.execute, sql, parameters)
  File "/home/vscode/.local/lib/python3.11/site-packages/aiosqlite/cursor.py", line 40, in _execute
    return await self._conn._execute(fn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/aiosqlite/core.py", line 133, in _execute
    return await future
           ^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/aiosqlite/core.py", line 106, in run
    result = function()
             ^^^^^^^^^^
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await <http://self.app|self.app>(scope, receive, _send)
  File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/gzip.py", line 24, in __call__
    await responder(scope, receive, send)
  File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/gzip.py", line 44, in __call__
    await <http://self.app|self.app>(scope, receive, self.send_with_gzip)
  File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await <http://self.app|self.app>(scope, receive, sender)
  File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await <http://self.app|self.app>(scope, receive, send)
  File "/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await <http://self.app|self.app>(scope, receive, send)
  File "/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/prefect/server/utilities/server.py", line 103, in handle_response_scoped_depends
    response = await default_handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py", line 241, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py", line 167, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/prefect/server/api/workers.py", line 529, in worker_heartbeat
    await models.workers.worker_heartbeat(
  File "/home/vscode/.local/lib/python3.11/site-packages/prefect/server/database/dependencies.py", line 119, in async_wrapper
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/prefect/server/models/workers.py", line 608, in worker_heartbeat
    result = await session.execute(insert_stmt)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/ext/asyncio/session.py", line 436, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 190, in greenlet_spawn
    result = context.throw(*sys.exc_info())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2225, in execute
    return self._execute_internal(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2120, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/bulk_persistence.py", line 1272, in orm_execute_statement
    result = conn.execute(
             ^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1412, in execute
    return meth(
           ^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 483, in _execute_on_connection
    return connection._execute_clauseelement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1635, in _execute_clauseelement
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1844, in _execute_context
    return self._exec_single_context(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1984, in _exec_single_context
    self._handle_dbapi_exception(
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
    self.dialect.do_execute(
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
    cursor.execute(statement, parameters)
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 113, in execute
    self._adapt_connection._handle_exception(error)
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 263, in _handle_exception
    raise error
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 95, in execute
    self.await_(_cursor.execute(operation, parameters))
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 125, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 185, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/aiosqlite/cursor.py", line 48, in execute
    await self._execute(self._cursor.execute, sql, parameters)
  File "/home/vscode/.local/lib/python3.11/site-packages/aiosqlite/cursor.py", line 40, in _execute
    return await self._conn._execute(fn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/aiosqlite/core.py", line 133, in _execute
    return await future
           ^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/aiosqlite/core.py", line 106, in run
    result = function()
             ^^^^^^^^^^
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: INSERT INTO worker (name, last_heartbeat_time, work_pool_id, id, created, updated) VALUES (:name, :last_heartbeat_time, :work_pool_id, :id, :created, :updated) ON CONFLICT (work_pool_id, name) DO UPDATE SET last_heartbeat_time = :param_1]
[parameters: {'name': 'ProcessWorker 554a8335-b13c-43c1-9b86-5055990e4226', 'last_heartbeat_time': '2023-06-14 21:16:16.078302', 'work_pool_id': '5625f1a6-7425-4494-ac1b-2bb64b10d3a6', 'id': 'e8c98f3b-89b2-4b78-a7c8-9f0ce4da7440', 'created': '2023-06-14 21:16:16.079493', 'updated': '2023-06-14 21:16:16.079519', 'param_1': '2023-06-14 21:16:16.078302'}]
(Background on this error at: <https://sqlalche.me/e/20/e3q8>)
Thanks Jeff. Restarting the worker did the trick
🙌 1
4 Views
Open in Slack
PreviousNext
Powered by