Hi everyone, I am migrating one of our core only runs to prefect server, but couldn’t get the tasks to work in parallel. The core only run uses

flow.run(executor=LocalDaskExecutor(scheduler="threads", num_workers=4))

and can execute tasks in parallel. The server registration is as follows:

flow.storage = Docker(....)
flow.run_config = DockerRun()
flow.executor = LocalDaskExecutor(scheduler="threads", num_workers=4)
flow.register(project_name=... , build=False)

The flow can run and communicate with prefect server, but the tasks are executed sequentially. Both the gannt chart and the logs show this. Am I missing something? (prefect==0.13.13)

Hello! We encountered something similar with the executor being ignored, but using

environment

instead worked

flow.environment = LocalEnvironment(executor=LocalDaskExecutor(scheduler="threads", num_workers=4))

I had initially tried environments, exactly the same way, didn’t work. Thats when I found out environments are deprecated, and substituted for run_config + executor. I am suspecting that my executor choices aren’t being picked up by server, and prefect is falling back to the default

LocalExecutor

. Can you (or anyone else) verify whether prefect server shows information about the executor for a registered flow? Either directly under the flow details, or under the flow environment details?

I did not see executor information in the flow details (or in the GraphQL responses) I've assigned the environment at the time of the flow initialization and not as a later assignment (and not the time of register), if that matters

Time of assignment was the issue 😅 . I was assigning server related members (

flow.storage,flow.run_config, flow.executor

) at register time, while building a custom docker image, and serializing my flow in the docker image before register time, in a previous run of python. Therefore, my serialized flow pickle had empty values for

flow.storage

and

flow.executor

. I suppose only

storage

and

run_config

are sent to server, and

executor

is not. Since prefect server was not enforcing an executor on my flow run, and my serialized flow had no executor choice, I was falling back to the default

LocalExecutor

. Many thanks, discussing with you probably saved me multiple hours 😊

@Marvin archive “Registering a Flow with Prefect Server isn’t respecting my executor configuration”