Avi A
07/07/2020, 9:05 AMResult
and I just want to run task C. In Luigi, C would ask for B’s output, and only fetch that. However, in prefect, the flow runner will fetch ALL the results of the mapped tasks A, but since we already have B’s result, that’s totally redundant and wastes a lot of time and data exchange.
Any idea on how to tackle this issue? i.e. fetch only task B results somehow so that C can run?Zachary Hughes
07/07/2020, 12:50 PMAvi A
07/07/2020, 1:04 PMZachary Hughes
07/07/2020, 1:11 PMAvi A
07/07/2020, 1:12 PMZachary Hughes
07/07/2020, 1:20 PMAvi A
07/07/2020, 1:25 PMstate
of Flow A?Zachary Hughes
07/07/2020, 1:35 PMquery {
flow_run (where: {flow_id: {_eq: "flow-a-id-here"}}){
state
}
}
You can play around with this via the interactive API, but if there's something you'd like to query programmatically, you can do so using Core's client.graphql
method.Avi A
07/07/2020, 1:44 PMvariables
when running a graphql? I mean, what’s the mark for a variable placeholder inside a query?Zachary Hughes
07/07/2020, 2:04 PMAvi A
07/07/2020, 2:10 PMvariables
argument in the client.graphql
method
prefect.client.client.Client.graphql(query, raise_on_error=True, headers=None, variables=None, token=None)
State
object that you get when running locally i.e.
state = flow.run()
get_task_run_info
I’m getting very general stuff. example:
>>> client.get_task_run_info("b598dc53-3494-4de4-ae5f-ca48e5b7b636", task_id=task_id)
Out[66]: TaskRunInfoResult(id='56a761f8-0143-47dd-b7d0-f19ae5d2f57a', task_id='cf92e3c2-7868-4910-83aa-826d550b2d89', task_slug='33990e35-d898-43ef-a596-a7242e5a31f3', version=2, state=<Success: "Task run succeeded.">)
Zachary Hughes
07/07/2020, 2:19 PMget_task_run_info
is essentially a canned query, so you'll want to specify your own fields. In this case, sounds like you'll want something like this:
query {
flow_run (where: {flow_id: {_eq: "your-id-here"}}){
task_runs {
states {
result
serialized_state
}
}
}
}
Avi A
07/07/2020, 2:21 PMtask_run
inside the query you just wroetZachary Hughes
07/07/2020, 2:21 PMAvi A
07/07/2020, 2:23 PMZachary Hughes
07/07/2020, 2:24 PMAvi A
07/07/2020, 2:24 PMMarwan Sarieddine
07/07/2020, 4:35 PMZachary Hughes
07/07/2020, 4:37 PMMarwan Sarieddine
07/07/2020, 4:37 PM