Hello! I'm having some trouble with running a flow on a single node dask cluster on Kubernetes. When I run the flow locally, the flow works as expected, but when I run it on Kubernetes , I get the following error:

Unexpected error: TypeError("Cannot map over unsubscriptable object of type <class 'NoneType'>: None...")

. The problems seems to come from a mapped task that outputs a pandas dataframe. Anyone has any ideas on how to fix this problem or how to debug it further?

Hi @Julien Allard Welcome! I don’t think we explicitly support mapping over pandas

DataFrame

s directly. If it works locally, that’s great. But, I think you’ll need to convert your

DataFrame

to a

List

. But let me double-check with the team!

Oops sorry my message might have been confusing, I'm actually mapping over a list of dataframes and not directly a dataframe

Ahh understood

No I don't have any results configured

I’d suggest starting there. Even though you’re running on a single node, it’s best not to assume any reliable infrastructure in the cloud. Having a

GCS

or

S3

(depending on your cloud provider or choice`) result should guarantee that upstream results are passed and read correctly and that your flow can retry from failure

Sounds like a good idea. I have also noticed in the logs that the flow seems to start multiple times. I see the log

Beginning Flow run for 'Flow Example'

multiple times. Could that be related?

What environment do you have configured?

I have a

DaskKubernetesEnvironment

Hmm

would that be the flag

scheduler_logs

?

If it doesn’t solve your issue ping me here and we can take some further debugging steps

Alright, I will try the result solution. Thanks for your help!

Anytime!