<@ULVA73B9P> Is there a way I can check which task...
# ask-marvin
d
@Marvin Is there a way I can check which task runner is being used within my flow? Can I print something in my python script to check this?
n
@Marvin oi
hmm - will look into that
you can always print the task runner out of the active flow run context, but im not sure why you might need to do that
d
Basically, I'm trying to use the Dask task runner, but instead of passing it into the @flow parameter directly (which was causing a pickling error that I've still been unable to resolve), I'm trying to pass it in later on and see if that helps. But right now I'm not confident that Prefect is actually using Dask since there's no activity on the Dask dashboard
n
like
with_options
? or how are you doing this?
I'm trying to pass it in later on
d
trying with
with_options
as well as
flow.task_runner=___
Not really sure, I'm gonna trying to figure out the pickling issue I've asked on ask-community
n
i would not recommend mutating state on the flow instance like this
Copy code
flow.task_runner=___
that is what
with_options
is for do you have an example where its not using the selected task runner reliably?
also if you have an MRE of the pickling thing, please share. apologies if you have previously
d
@Nate Here's what I sent in the other channel
Has anyone had any issues with pickle/serialization? I'm getting the following error from a flow failure (note that
square
is the name of the task that I'm passing to my flow...
Copy code
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/distributed/protocol/pickle.py", line 60, in dumps
    result = pickle.dumps(x, **dump_kwargs)
_pickle.PicklingError: Can't pickle <function square at 0x7b68d25cd1c0>: it's not the same object as __main__.square
For more context, I'm using the dask task_runner with a pre-existing Fargate cluster I have on AWS ECS. From my understanding prefect serializes tasks prior to sending them to the dask scheduler, and that might be where my issue is. Not quite sure where to look/what to fix in my code regarding this though. Thanks!