Hello Everyone, I'm facing an issue that can't be figured by me. In a flow, the tasks are going to pending state instead of running. There is delay of a few seconds between tasks which is affecting my complete flow execution time. Note: I haven't introduced any retries or delays. My concurrency limit is unlimited.

Hey, do you have an minimum reproducible example that illustrates the behavior youre seeing?

@Jake Kaplan, Thanks for your reply. Not sure about the example as I'm not having any issues with the same tasks in another flow. But all runs of this flow have the same issue of delay. You can check my screenshot which shows pending state issue as well as the time difference between states.

I can see the pending task. Do you have a flow run that gets put into this state reliably? it's hard to tell what's going on w/o understanding more of the code/your setup 🙂

Okay Jake, I'll share the simple version of my code. My flow run is not coded to put the tasks to pending.

Are you passing a lot of data between tasks when you see this jump? I can you try wrapping the data passed to the task that seems delayed with

quote

? example below:

from prefect import flow, task
from prefect.utilities.annotations import quote

@task
def my_task(a):
    ...

@flow
def my_flow():
   data = 5
   my_task(quote(data))

Yes I'm pasasing around 20K+ records. I'll try using this quote

in order to show parameters in the UI, prefect needs to look at the parameters structure.

quote()

should keep that from happening?

Using quote also had the gap between tasks

my apologies, it seems like this is still an open issue: https://github.com/PrefectHQ/prefect/issues/9934

Yes

i'm suprised that it looks like it's taking longer w/ quote? It should have cut a good % of the time off based on the issue, but likely not all of it

The difference I noticed is, Before it used to show 1s for the extract2 task and there was a huge delay between extract2 and check_indb_serp (2m 45s) but after using quote extract2 shows 1m 3s and the delay time is a little less(38s). FYI, this difference is observed using the same data.

This should be resolved in the next release, likely put out on thursday, https://github.com/PrefectHQ/prefect/pull/10370 you'll be able to use

quote()

to avoid introspection of large task run parameters

Thanks @Jake Kaplan