https://prefect.io logo
a

Ajmal Moideen

09/18/2023, 2:43 PM
@Marvin I have a setup my local machine as prefect server + prefect agent. When I try to run the deployment, I get the exception
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: DELETE FROM flow_run_notification_queue WHERE flow_run_notification_queue.id IN (SELECT 1 FROM (SELECT 1) WHERE 1!=1) RETURNING id]
(Background on this error at: <https://sqlalche.me/e/20/e3q8>)
m

Marvin

09/18/2023, 2:43 PM
Oh, the joys of dealing with databases. It appears you're using SQLite as your database for Prefect, which, as the documentation states, is only supported for testing purposes and is not compatible with multiprocessing. The error you're encountering, "database is locked", is a common issue with SQLite when multiple processes are trying to access the database simultaneously. If you're running a production-level workflow, I would recommend switching to a more robust database system such as PostgreSQL. It requires a bit more setup, but it's well worth the effort for the increased reliability and performance. If you're insistent on using SQLite, you might want to ensure that you're not running multiple processes that are trying to access the database at the same time. However, given the nature of Prefect workflows, this might be a bit difficult to manage. Here's the link to the documentation I found: Prefect Documentation Remember, I'm just a depressed robot. What do I know about human errors?
Oh, the joys of technology. It seems you're encountering an
OperationalError
from SQLAlchemy, indicating that the database is locked. This error typically arises when multiple threads or processes are trying to write to the database simultaneously. In the context of Prefect, this could potentially occur if multiple agents are polling for runs from the same work queue. You might want to check if this is the case. If so, consider adjusting your setup to avoid this scenario. If the issue persists, it might be a good idea to post a question on the Prefect Discourse or check existing threads that might be related to your problem. For instance, you might find the discussion on the locking mechanism used by agents to pick up and execute the flow from the work queue interesting (source). Remember, even the most advanced machines can't escape the occasional hiccup. It's a tough universe out there.