https://prefect.io logo
Join Slack
Powered by
Hi there ! Go some weird issue trying out flows lo...
# prefect-community
s
Hi there ! Go some weird issue trying out flows locally. I'm running a local orion with 
prefect orion start --port 5000
as well as the very simple flow copied below. Running the flow with 
python log_flow.py
randomly ends up crashing with this error : 
RuntimeError: The connection pool was closed while 2 HTTP requests/responses were still in-flight
. Is this an issue related to the server ? Am I forgetting to await something ? Anything I could do to fix it ?
👀 1
Copy code
import prefect
from prefect import flow, task, get_run_logger
from utilities import AN_IMPORTED_MESSAGE


@task
def log_task(name):
    logger = get_run_logger()
    <http://logger.info|logger.info>("Hello %s!", name)
    <http://logger.info|logger.info>("Prefect Version = %s 🚀", prefect.__version__)
    logger.debug(AN_IMPORTED_MESSAGE)


@task(description="get_names")
def get_names():
    return range(50)


@flow()
def log_flow():
    names = get_names()

    for name in names:
        log_task.submit(name)


if __name__ == "__main__":
    log_flow()
p
Can you share info about your environment and settings? I just ran this in a loop 100 times on my machine without a crash, but I'm trying to reproduce exactly this error for a bug that I've seen reported.
s
Sure ! Using Python 3.8.12, Prefect 2.6.9, httpx 0.23.1, httpcore 0.16.1, anyio 3.6.2. Computer is using Ubuntu 20.04.
Any more information could be of use to you ?
I initially had the issue with a more complex flow and managed to narrow the complexity down to this so it definitely looks related to the environment, yeah
upvote 1
❤️ 1
p
Which database and prefect API are you using?
s
Whatever the default is, I suppose
👍 1
p
Awesome - that simplifies things even further 🙂
s
Started by following the getting started tutorial from the docs
👍 1
Virtualenv was created by pyenv and dependency management is done with Poetry (1.1.11) if that is somehow relevant
🙏 1
p
How frequently are you getting this error?
s
Hard to say. I'd say it happens about 3 times out of 4 when running the flow, although the number of in-flight requests varies from time to time.
p
Hmmmm. I'm on • ubuntu VM running 22.04 on an M1 Mac using Parallels • python 3.8.15 • httpx 0.23.1 • httpcore 0.16.1 • anyio 3.6.2 • prefect 2.6.9 and can't reproduce.
@Mathijs Carlu Are you experiencing the same issue, can you reproduce with the same minimal example, and what are you running?
s
Won't have time today but I could try to see if reproducing works in a Docker image ? That might make things easier for you.
p
That would be incredible
s
I'll keep you posted tomorrow morning most likely (UTC+1)
🙏 1
m
@Peyton Runyan my problem and setup is shared here. The error I see has been varying. I've seen the one in the shared thread, seen the one in this thread, seen 500 server errors, seen RemoteProtocolErrors (probably some others as well). Mostly, it's one of the mapped tasks that can't manage to report its state to the API so it crashes. The flow continues however, and the task depending on it tries to read its state but can't. I suspect some correlation in all these errors. I could try running the same example, but I don't know when I'll have time for this.
p
Thanks!
@Sylvain Hazard can you upgrading from 
httpcore 0.16.1
to 
0.16.2
and see if that addresses it? Apparently there's an issue with it that has been causing people trouble elsewhere.
Can you also include a full stack trace the next time you get that crash?
s
I'm home from work already, I'll try it out in the morning. Thanks for digging in !
👍 1
Hey ! It does seem to be working very well, weird that it doesn't happen on your environment. Would you like me to make a PR adding httpcore >= 0.16.2 to the requirements ?
p
That’s great to hear! I’d normally say yes, but I want to check with @Zanie first and see if we just want to avoid 0.16.1 specifically vs forcing 0.16.2 and above.
s
Right, makes sense. Should I try to make a Docker image that reproduces the issue for further investigation ?
p
Probably not at this point. It looks like the 
connection pool closed with xyz in-flight
is always caused by some sort of upstream error, and this case the error is httpcore. Thanks a ton for working with us on this thought man!
s
My pleasure, thanks a lot for your time !
❤️ 1
z
see if we just want to avoid 0.16.1 specifically vs forcing 0.16.2 and above
Yeah I believe that is better practice.
👍 1
Open in Slack
PreviousNext
Powered by