Hi, I try to run Prefect Server on Docker box. All...
# prefect-server
s
Hi, I try to run Prefect Server on Docker box. All services are started and running and I can registry my flow and run from UI. But when it runs it uses LocalExceutor/ How can I make it to use Dask? I added DaskExecutor directly to the flow but it still runs with Local. I also tried to set env var PREFECT__ENGINE__EXECUTOR__DEFAULT_CLASS in docker-compose.yml file and in my flow but it did not help. How to make Prefect Server use Dask as executors while running flows? Update with code. Flow creation: def create_flow(name):   executor = get_executor(env='dev')   with Flow(name,           storage=Local(),           executor=executor) as flow:       input = Parameter('input_params', required=True)       data1 = _task1(input)       data2 = _taks2.map(data1)       data3 = _task3.map(data2)       data4 = _task4.map(data3)       result = _task5(data4)   return flow get_executor function returns DaskExecutor Flow registration: 
Copy code
flow = create_flow('TestFlowServer')
flow.register(project_name="Test")
   docker-compose.yml Added below var to environment section for towel and graphql services PREFECT__ENGINE__EXECUTOR__DEFAULT_CLASS: prefect.engine.executors.DaskExecutor I use LocalAgent to run flows. I also added the above env var when I start it. But nothing helped. When I try to run the flow locally, without registering on Server everything works. Thanks, Sergey
d
Hi @Sergiy Krutsenko! Would you mind sharing your Flow code here? Specifically I’m interested in: 1. Your Storage 2. Your environment / run config It would also be helpful to know your agent type (Docker, Local, Kubernetes, etc)
s
Hi @Dylan Here it is. Also edited my original post Flow creation:   def create_flow(name):   executor = get_executor(env='dev')   with Flow(name,           storage=Local(),           executor=executor) as flow:       input = Parameter('input_params', required=True)       data1 = _task1(input)       data2 = _taks2.map(data1)       data3 = _task3.map(data2)       data4 = _task4.map(data3)       result = _task5(data4)   return flow   get_executor function returns DaskExecutor   Flow registration: 
Copy code
flow = create_flow('TestFlowServer')
flow.register(project_name="Test")
   docker-compose.yml   Added below var to environment section for towel and graphql services PREFECT__ENGINE__EXECUTOR__DEFAULT_CLASS: prefect.engine.executors.DaskExecutor   I use LocalAgent to run flows. I also added the above env var when I start it. But nothing helped.   When I try to run the flow locally, without registering on Server everything works.    Thanks, Sergiy
d
Would you mind sharing your
get_executor
function?
Also, Prefect Server is the orchestration layer of the stack and isn’t responsible for how your flow runs, that’s defined on the flow object itself. So setting that environment variable in the Prefect Server services shouldn’t affect execution
You may need to configure a
LocalRun
for this flow. Otherwise, I believe the Flow looks for a run config from the agent
s
@Dylan I fund another solution. I use RemoteDaskEnvironment with flow and it works now.
d
Glad to hear it!