https://prefect.io logo
Title
c

Chris Goddard

05/26/2023, 6:08 PM
I’m still running into a lot of issues with async flows — raised this issue 10 days ago specifically related to subflows - but having massively stripped down the amount I’m using prefect (basically just using it to run the
main
function - with everything else being plain async python) and i’m still getting random hangs — and when I kill the script, it seems (as best as I can tell), that the
KeyboardInterrupt
gets caught somewhere within the prefect codebase. what is curious is that when I dropped the top-level
@flow
altogether and just ran as a plain python script, it hasn’t stalled yet (will update if it does). is it possible that there’s something going on with longer-running (5min+) async flows that’s causing them to hang at some point? I thought maybe it was to do with hitting the logging rate limit but by removing all of the task decorators I’ve managed to limit the overall prefect calls significantly. I’m racking my brain at this point. anyone have any experience with long-running async programs? are there other factors that could be causing random hangs?
z

Zanie

05/26/2023, 6:21 PM
Can you share the keyboard interrupt traceback?
And the version you’re using?
It’s also very useful to get a
py-spy dump
of the process when it’s stuck
o

Ouail Bendidi

05/26/2023, 7:15 PM
Hey @Chris Goddard I've noticed the same issue when using docker container infra on a small enough prefect agent, maybe this thread will give you more context: https://prefect-community.slack.com/archives/CL09KU1K7/p1683203799355429?thread_ts=1683203799.355429&cid=CL09KU1K7