https://prefect.io logo
Join Slack
Powered by
Hi here the tasks on Left branch are running fine ...
# ask-community
m
Hi here the tasks on Left branch are running fine but the tasks on the Right Branch are stuck in Mapped state. Both have the same functions called and mapped to run a task.
k
Are you using LocalDaskExecutor?
m
Yes LocalDaskExecutor
k
Are you using threads or processes? Processes might help it become more stable
m
Copy code
left_branch = processing.map(processing_channels([('some_var1', date)]))
right_branch =  processing.map(processing_channels([('some_var2', date)]))
k
No no I mean 
LocalDaskExecutor(scheduler="processes")
m
flow.executor = LocalDaskExecutor()
k
Yes the default is 
threads
. 
processes
is more stable. You can see the syntax here
m
Thanks!
Also is there any maximum limit for parallel processing?
k
Not on the Prefect side. It would be your hardware limits
m
Cool.
Hi @Kevin Kho i am still facing the Mapped state issue even after adding processes in LocalDaskExecutor
k
Does it not run after the left side finishes running? Prefect currently only runs one mapped task at a time
m
Yes but the left side takes several hours to finish. I want to run both mapped task to run in parallel as they are not interdependent.
k
Ah I see now. The FlowRunner currently only runs one mapped task at a time.
m
Ohh okay, let me know if you find any fix. Thanks!!
Open in Slack
PreviousNext
Powered by