https://prefect.io logo
#prefect-community
Title
# prefect-community
m

Minakshi

08/28/2020, 10:47 PM
1. Is there a way to pass parameters to via dask executor to flow? The requirement here is to run a flow for different datasets which could be configured either from dask or the caller. And the execution of flows of different datasets could be running in parallel. 2. Is there a way to start the run of a flow using a Flask api? and also can the cancellation of flow be controlled by an api? (I tried doing this but i was getting mutiprocessor errors)
n

nicholas

08/28/2020, 10:52 PM
Hi @Minakshi - the answer to your first question is "sort-of" and ties into your second question. The reason it's not resoundingly yes is that executors aren't currently configured at runtime, which means you register a flow with an executor context that's read at runtime but isn't modified. That said, flow runs can be launched from any client with access to GraphQL (including
prefect.Client
), which means you can use a Flask or any other API to kick off a flow run, provided it has network access to the API (whether Server or Cloud).
For example, you could do this with the
prefect.Client
with the following code:
Copy code
import prefect
client = prefect.Client()

client.create_flow_run(flow_id="your_flow_id", parameters={"some_parameters": "that_change_per_run"})
m

Minakshi

08/28/2020, 11:07 PM
will this run synchronously or asynchronously?
n

nicholas

08/28/2020, 11:08 PM
These are HTTP request wrappers, so would be run asynchronously
m

Minakshi

08/28/2020, 11:08 PM
the flow i am working with is time consuming and has subprocess calls, if it synchronous, it blocks the current thread and hence the http request goes in a limbo..
will try it out. thanks
n

nicholas

08/28/2020, 11:09 PM
Ah sorry - are you using just Prefect Core? Or are you using Server/Cloud?
m

Minakshi

08/28/2020, 11:09 PM
prefect core
n

nicholas

08/28/2020, 11:11 PM
Got it - in which case disregard the advice I gave above;
flow.run()
is a synchronous call, which will lock your current thread but will let you pass in parameters and an executor at runtime.
m

Minakshi

08/28/2020, 11:12 PM
hmm ok
n

nicholas

08/28/2020, 11:12 PM
In order to make this happen asynchronously (or without blocking the main thread), you'll need to use normal Pythonic ways of spinning off new threads
Your final question about cancellation will require that you have some sort of infrastructure/API setup, which would mean you need a persistence layer like Prefect Server/Prefect Cloud
m

Minakshi

08/29/2020, 12:10 AM
okay
2 Views