Mini Khosla

05/05/2022, 8:51 AM
Hi All We are using prefect server 0.15.7 and running the flows on yarn Cluster. I have 20 mapped tasks and each mapped task loops over a list , does some processing and saves output to hadoop. I am getting the below error intermittently. Any help would be appreciated.

Anna Geller

05/05/2022, 10:28 AM
Are you running Dask on a Hadoop cluster? The error says something about unmanaged history and it seems like a Dask issue. Can you share the flow code for this flow (mainly interested in flow structure and configuration of Flow object, not what tasks are doing)? What agent did you use?

Kevin Kho

05/05/2022, 1:49 PM
The KilledWorker indicates you have some kind of package mismatch between Client/Scheduler/Workers
:upvote: 1