Jai P
08/22/2022, 11:51 PMsqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
details in the thread!parent_flow
• the parent_flow
kicks of subflow_1
, subflow_2
asynchronously
• subflow_1
has no dependencies, but subflow_2
depends on the result of subflow_1
, so it triggers subflow_1
as well
• ** i don't want to have to topologically sort all my subflows to figure out the optimal execution order, i'd rather rely on prefect caching to minimize workRyan Peden
08/23/2022, 3:27 PMJai P
08/23/2022, 3:30 PMIn-memory databases are only supported by Prefect Orion for testing purposes and are not compatible with multiprocessing.derp i completely missed this! i'll try using postgres and update if i see it working 🙂
Ryan Peden
08/23/2022, 3:32 PMZanie
08/23/2022, 4:50 PMJai P
08/23/2022, 4:56 PMZanie
08/23/2022, 5:01 PMJai P
08/23/2022, 5:02 PM@task
def test_task():
print("hi")
@flow
def test_flow():
with tags("testing"):
test_task()
if __name__ == '__main__':
test_flow()
isn't showing tags in the orion ui when using postgres, but does show them when using sqliteZanie
08/23/2022, 5:38 PMJai P
08/23/2022, 5:47 PMwith tags(...):
block in the wrong place, and it should be around the asyncio.gather
call instead:
# not this
subflows = []
with tags("some_tag"):
subflows.append(my_subflow())
asyncio.gather(*subflows)
# do this instead
subflows = []
subflows.append(my_subflow())
with tags("some_tag"):
asyncio.gather(*subflows)