Channels
#prefect-community
Title
# prefect-community
m
Minakshi
08/28/2020, 10:47 PM1. Is there a way to pass parameters to via dask executor to flow? The requirement here is to run a flow for different datasets which could be configured either from dask or the caller. And the execution of flows of different datasets could be running in parallel.
2. Is there a way to start the run of a flow using a Flask api? and also can the cancellation of flow be controlled by an api? (I tried doing this but i was getting mutiprocessor errors)
n
nicholas
08/28/2020, 10:52 PM
Hi @Minakshi - the answer to your first question is "sort-of" and ties into your second question. The reason it's not resoundingly yes is that executors aren't currently configured at runtime, which means you register a flow with an executor context that's read at runtime but isn't modified. That said, flow runs can be launched from any client with access to GraphQL (including
prefect.ClientFor example, you could do this with the
prefect.Client Copy code
import prefect
client = prefect.Client()
client.create_flow_run(flow_id="your_flow_id", parameters={"some_parameters": "that_change_per_run"})More on the
Clientm
Minakshi
08/28/2020, 11:07 PMwill this run synchronously or asynchronously?
n
nicholas
08/28/2020, 11:08 PM
These are HTTP request wrappers, so would be run asynchronously
m
Minakshi
08/28/2020, 11:08 PMthe flow i am working with is time consuming and has subprocess calls, if it synchronous, it blocks the current thread and hence the http request goes in a limbo..
will try it out. thanks
n
nicholas
08/28/2020, 11:09 PM
Ah sorry - are you using just Prefect Core? Or are you using Server/Cloud?
m
Minakshi
08/28/2020, 11:09 PMprefect core
n
nicholas
08/28/2020, 11:11 PM
Got it - in which case disregard the advice I gave above;
flow.run()m
Minakshi
08/28/2020, 11:12 PMhmm ok
n
nicholas
08/28/2020, 11:12 PM
In order to make this happen asynchronously (or without blocking the main thread), you'll need to use normal Pythonic ways of spinning off new threads
Your final question about cancellation will require that you have some sort of infrastructure/API setup, which would mean you need a persistence layer like Prefect Server/Prefect Cloud
m
Minakshi
08/29/2020, 12:10 AMokay
2 Views