We are mapping over a series of requests, to make ...
# prefect-community
j
We are mapping over a series of requests, to make multiple tasks within the same flow, • We want to set the task state to be failed/pending for each task, after it has run. • Then later we want to update the task state to be succeeded via the graphQL/rest API • We are using Prefect cloud as the API back_end. ---- • Currently what python methods/GraphQL should we use to update the state of individual task runs inside of flow runs?
a
Currently what python methods/GraphQL should we use to update the state of individual task runs inside of flow runs?
You could do that using Prefect client:
Copy code
from prefect import Client
client = Client()
results = client.graphql(query)
and query could me either query or a mutation e.g. to change the task run states
you could even do that without a query or mutation, but using this client's method
j
Thank you ❤️
🙌 1
@Anna Geller is the retry still running a in a synchronous thread?
We might have to wait for up to 1 hour for the response and we have 1000's of these tasks.
Will a retry lock up the compute or not?
a
Retries don't create separate task runs; they instead update the run history of the failed task run. But each task run is independent, they don't lock each other's compute. For example, if you run it with LocalDaskExecutor, each task run can run in its own thread or process - does it answer your question?
j
That makes sense 🙂
Also, please how do I get all of the
task_runs ids
from one map function downstream into another map task? 🙏
a
you don't actually need to explicitly retrieve the task run IDs. Instead, you can do iterated mapping as shown here this created independent mapped pipelines
Copy code
from prefect import Flow, task

@task
def add_ten(x):
    return x + 10

with Flow('iterated map') as flow:
    mapped_result = add_ten.map([1, 2, 3])
    mapped_result_2 = add_ten.map(mapped_result)
j
👍