https://prefect.io logo
Title
j

Jackson Maxfield Brown

06/22/2020, 7:30 PM
What are the reasons that a task would finish with a "finished task run for task with final state: 'Pending'"? My pipeline runs and all tasks succeed up until one of the tasks return with a "Pending" status. Then after this, the system says: "Flow run RUNNING: terminal tasks are incomplete" and then "Beginning Flow run for ...." but then nothing else is printed and it looks like its just hanging there. I have let the system sit there for a long while but it really seems like its hanging and not waiting for the Pending tasks. Any ideas?
k

Kyle Moon-Wright

06/22/2020, 7:54 PM
Hmm, I haven't seen a task run ending in a Pending state before - just awaiting execution. Following this list of state transitions, this Pending state should resolve into a Scheduled or Running state (or eventual subclass Failure) based on your task configurations, but it's not even making it there. I wonder if there is some task run logic that needs to be resolved to move this task run to a Running state.
Can you share the FlowRunner/TaskRunner output?
j

Jackson Maxfield Brown

06/22/2020, 7:58 PM
The log is very long but here you go:
prefect log
Oh wait
thats not even all of it
but the important "Pending" result is there
My general idea is that it may be a bug? The
get_minutes_item_file_details
task is has a
retry
and a
retry_delay
attached so maybe the map "finishes" and sends the signal to the downstream to move on but the downstream checks for all data and sees that there were a couple of failures that are retrying? Just an idea and not sure how the internals of Prefect work too deeply.
k

Kyle Moon-Wright

06/22/2020, 8:08 PM
Very strange - the task ends in a 'Pending' state almost immediately. I'm going to do some more digging but this may be best resolved as a Github issue, for visibility with the development team.
j

Jackson Maxfield Brown

06/22/2020, 8:10 PM
Cool I can open one in a bit. With all the details. The flow is entirely runnable by the public too so the dev team can try that as well if they so choose
👍 1
k

Kyle Moon-Wright

06/22/2020, 8:14 PM
Awesome, thank you for the context. It does look like this may be a result of the mapping with
get_minutes_item_file_details
as you mentioned, which recently underwent a refactor to accommodate DFE - especially considering you are using the
DaskExecutor
.
j

Jackson Maxfield Brown

06/22/2020, 8:26 PM