https://prefect.io logo
Title
a

Austin Mackillop

05/20/2021, 6:44 PM
Hello, is there a way to have the prefect agent execute a flow in a single threaded manor? I converted an existing ml model training script that makes use of some datatypes that cannot be pickled. Returning these types from tasks works fine when executing the flow locally in a single thread but using the DaskExecutor or running the flow on the agent causes the flow to fail.
k

Kevin Kho

05/20/2021, 6:45 PM
Hey @Austin Mackillop, you can explicitly specify LocalExecutor, which is the default (or maybe even LocalDaskExecutor). I’ve seen this some for some people. What
Storage
are ytou using?
a

Austin Mackillop

05/20/2021, 6:47 PM
I tried specifying the executor like so:
with Flow("train-model", executor=LocalExecutor()) as flow:
But on the Ui it still appears to be doing things in parallel. I’m using whatever the default storage is.
k

Kevin Kho

05/20/2021, 6:51 PM
Could you run your flow with debug level logs and see the executor that shows when the flow starts?
a

Austin Mackillop

05/20/2021, 7:06 PM
How do I do that with the local agent?
k

Kevin Kho

05/20/2021, 7:08 PM
One sec let me try
Add
flow.run_config = LocalRun(env={'PREFECT__LOGGING__LEVEL':'DEBUG'})
to the flow code
a

Austin Mackillop

05/21/2021, 9:22 PM
I changed my tasks such that none of them returned un serializable objects and that solved my problem
k

Kevin Kho

05/21/2021, 9:23 PM
Glad you got it working @Austin Mackillop!