https://prefect.io logo
n

navya magisetty

08/03/2023, 11:02 AM
Hello Everyone, I'm facing an issue that can't be figured by me. In a flow, the tasks are going to pending state instead of running. There is delay of a few seconds between tasks which is affecting my complete flow execution time. Note: I haven't introduced any retries or delays. My concurrency limit is unlimited.
j

Jake Kaplan

08/03/2023, 12:19 PM
Hey, do you have an minimum reproducible example that illustrates the behavior youre seeing?
n

navya magisetty

08/03/2023, 12:53 PM
@Jake Kaplan, Thanks for your reply. Not sure about the example as I'm not having any issues with the same tasks in another flow. But all runs of this flow have the same issue of delay. You can check my screenshot which shows pending state issue as well as the time difference between states.
j

Jake Kaplan

08/03/2023, 7:27 PM
I can see the pending task. Do you have a flow run that gets put into this state reliably? it's hard to tell what's going on w/o understanding more of the code/your setup 🙂
n

navya magisetty

08/04/2023, 8:47 AM
Okay Jake, I'll share the simple version of my code. My flow run is not coded to put the tasks to pending.
j

Jake Kaplan

08/04/2023, 3:30 PM
Are you passing a lot of data between tasks when you see this jump? I can you try wrapping the data passed to the task that seems delayed with
quote
? example below:
Copy code
from prefect import flow, task
from prefect.utilities.annotations import quote

@task
def my_task(a):
    ...

@flow
def my_flow():
   data = 5
   my_task(quote(data))
n

navya magisetty

08/04/2023, 5:26 PM
Yes I'm pasasing around 20K+ records. I'll try using this quote
j

Jake Kaplan

08/04/2023, 5:28 PM
in order to show parameters in the UI, prefect needs to look at the parameters structure.
quote()
should keep that from happening?
n

navya magisetty

08/04/2023, 5:34 PM
Using quote also had the gap between tasks
j

Jake Kaplan

08/04/2023, 5:36 PM
my apologies, it seems like this is still an open issue: https://github.com/PrefectHQ/prefect/issues/9934
which lines up with what you're seeing I believe
n

navya magisetty

08/04/2023, 5:41 PM
Yes
So there is no solution as of now😶
j

Jake Kaplan

08/04/2023, 5:54 PM
i'm suprised that it looks like it's taking longer w/ quote? It should have cut a good % of the time off based on the issue, but likely not all of it
n

navya magisetty

08/04/2023, 6:00 PM
The difference I noticed is, Before it used to show 1s for the extract2 task and there was a huge delay between extract2 and check_indb_serp (2m 45s) but after using quote extract2 shows 1m 3s and the delay time is a little less(38s). FYI, this difference is observed using the same data.
j

Jake Kaplan

08/14/2023, 7:43 PM
This should be resolved in the next release, likely put out on thursday, https://github.com/PrefectHQ/prefect/pull/10370 you'll be able to use
quote()
to avoid introspection of large task run parameters
n

navya magisetty

08/16/2023, 6:13 AM
Thanks @Jake Kaplan