1 Is there a way to pass parameters to via dask executor to

Question

1 Is there a way to pass parameters to via dask executor to flow The requirement here is to run a flow for different datasets which could be configured either from dask or the caller And the execution of flows of different datasets could be running in parallel 2 Is there a way to start the run of a flow using a Flask api and also can the cancellation of flow be controlled by an api I tried doing this but i was getting mutiprocessor errors

nicholas · Answer

Hi < Minakshi> the answer to your first question is sort of and ties into your second question The reason it s not resoundingly yes is that executors aren t currently configured at runtime which means you register a flow with an executor context that s read at runtime but isn t modified That said flow runs can be launched from any client with access to GraphQL including `prefect Client` which means you can use a Flask or any other API to kick off a flow run provided it has network access to the API whether Server or Cloud

nicholas · Answer

For example you could do this with the `prefect Client` with the following code ```import prefect client = prefect Client client create flow run flow id= your flow id parameters= some parameters that change per run ```

nicholas · Answer

More on the `Client` here <https docs prefect io api latest client client html client 2>

Minakshi · Answer

will this run synchronously or asynchronously

nicholas · Answer

These are HTTP request wrappers so would be run asynchronously

Minakshi · Answer

the flow i am working with is time consuming and has subprocess calls if it synchronous it blocks the current thread and hence the http request goes in a limbo

Minakshi · Answer

will try it out thanks

nicholas · Answer

Ah sorry are you using just Prefect Core Or are you using Server Cloud

Minakshi · Answer

prefect core

nicholas · Answer

Got it in which case disregard the advice I gave above `flow run ` is a synchronous call which will lock your current thread but will let you pass in parameters and an executor at runtime

Minakshi · Answer

hmm ok

nicholas · Answer

In order to make this happen asynchronously or without blocking the main thread you ll need to use normal Pythonic ways of spinning off new threads

nicholas · Answer

Your final question about cancellation will require that you have some sort of infrastructure API setup which would mean you need a persistence layer like

Minakshi · Answer

okay