kwmiebach
09/02/2022, 2:46 PMsqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140193261803264 and this is thread id 140192735962880.
I can also paste the part of the code where sqlite is called.
I am also trying to guess the reason behind the error. There is an sqlite object created in a function. Within the function I define another function which uses this object. But both python functions are not prefect flows or tasks. They live within a bigger prefect flow. So here comes my first question: Does prefect 2 create a different thread for each nested function inside a flow or a task. Otherwise I cannot explain why the 2 parts of the code would run in different threads.Khuyen Tran
09/02/2022, 2:53 PMkwmiebach
09/02/2022, 2:59 PMKhuyen Tran
09/02/2022, 3:01 PMRyan Peden
09/02/2022, 3:03 PMkwmiebach
09/02/2022, 3:04 PMnonlocal
keyword.Ryan Peden
09/02/2022, 3:08 PMkwmiebach
09/02/2022, 3:09 PMRyan Peden
09/02/2022, 3:12 PMConcurrentTaskRunner
, which uses a thread pool to run tasks. Your nested function creates a closure around the sqlite3.Row
created on the main thread, so you see this error when you execute the function later on inside a taskkwmiebach
09/02/2022, 3:14 PMSequentialTaskRunner
also uses threads? but in sequence?DB_FIBU
as a normal parameterRyan Peden
09/02/2022, 3:24 PMSeqentialTaskRunner
and see if the problem goes away. Again, it might not be the solution you want to use for your production flow but would provide useful information.kwmiebach
09/02/2022, 3:31 PMSequentialTaskRunner
the whole time now. When I comment the 'task' line, where the pipeline is executed, it works. When I uncomment it, which means the pipeline is executed in a prefect 2 task, then the 'different threads' error re-appears.Ryan Peden
09/02/2022, 3:48 PMkwmiebach
09/02/2022, 4:04 PM