has anyone else used LocalDaskExecutor in k8s? It seems the dask parent management process does not actually work under k8s. I'm getting all sorts of wacky errors. If it is not supported, it might be an anti-pattern, parallel processes inside k8s for prefect?
z
Zanie
04/12/2021, 3:09 PM
Hey @Rob Fowler -- can you share some of the errors you're encountering? This should work just fine afaik.
r
Rob Fowler
04/13/2021, 6:41 AM
I'll make a simple example tomorrow. It basically does not work under an agent. In the meantime I am doing single threaded flows.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.