Is it possible to trigger multiple concurrent Flow...
# prefect-community
e
Is it possible to trigger multiple concurrent Flow Runs for the same Flow? I’m getting this error when I attempt to do so:
Copy code
RuntimeError: The task runner is already started!
Context: I have an application hosted on GCP Cloud Run that allows multiple requests per container. Whenever my application receives an API request with a
user_id
param, it triggers a flow run to update data for that user. When I trigger the first Flow Run via an API request, everything works fine. But if I call the API again while the first Run is ongoing, my second API request with throw the above RuntimeError. More context: My application is written in Flask and served with Gunicorn. It’s deployed on GCP Cloud Run. Here’s the gunicorn config in my dockerfile
Copy code
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 pelm_utility_service:app
1
I believe I could fix this by only allowing 1 request per worker and scaling workers when I receive more requests. But this is not ideal for performance reasons (time required to start each new worker).
r
I believe you could also work around this by using a separate task runner instance for each flow run. So, something like:
Copy code
flow_run = my_flow_function.with_options(task_runner=ConcurrentTaskRunner())
flow_run()
e
Ah yes, that seems to solve this. Thanks!
r
You're welcome!