Sorry for mentioning this again but I m still grappling with

Question

Sorry for mentioning this again but I m still grappling with this error Does this look familiar to anyone `ERROR asyncio Fatal error on SSL transport`

Wilson Bilkovich · Answer

``` python demo flow py 2021 08 27 14 32 43 0400 INFO prefect S3 | Uploading script demo flow py to prefect flows demo random sample wlb in addepar research data Flow URL ID 74703757 162b 4cc8 a3d6 2d61f13be9d3 Project demo Labels 2021 08 27 14 32 46 0400 INFO prefect FlowRunner | Beginning Flow run for increment a random sample 2021 08 27 14 32 46 0400 INFO prefect DaskExecutor | Creating a new Dask cluster with `dask kubernetes core KubeCluster` Creating scheduler pod on cluster This may take some time 2021 08 27 14 33 07 0400 INFO prefect DaskExecutor | The Dask dashboard is available at INFO prefect DaskExecutor The Dask dashboard is available at Users wilson bilkovich pyenv versions 3 9 6 envs addemart lib python3 9 site packages distributed client py 1105 VersionMismatchWarning Mismatched versions found + + + + + | Package | client | scheduler | workers | + + + + + | blosc | None | 1 10 2 | None | | dask | 2021 08 0 | 2021 08 1 | None | | distributed | 2021 08 0 | 2021 08 1 | None | | lz4 | None | 3 1 3 | None | + + + + + warnings warn version module VersionMismatchWarning msg 0 warning 2021 08 27 14 33 26 0400 INFO prefect FlowRunner | Flow run SUCCESS all reference tasks succeeded INFO prefect FlowRunner Flow run SUCCESS all reference tasks succeeded ERROR asyncio Unclosed client session client session lt aiohttp client ClientSession object at 0x11da37700 gt ERROR asyncio Unclosed client session client session lt aiohttp client ClientSession object at 0x11da373d0 gt ERROR asyncio Unclosed connector connections lt aiohttp client proto ResponseHandler object at 0x11dab9b20 gt 45 688264346 connector lt aiohttp connector TCPConnector object at 0x11da379a0 gt ERROR asyncio Unclosed connector connections lt aiohttp client proto ResponseHandler object at 0x11db50a00 gt 45 813355617 connector lt aiohttp connector TCPConnector object at 0x11da37ac0 gt ERROR asyncio Fatal error on SSL transport protocol lt asyncio sslproto SSLProtocol object at 0x11da62f10 gt transport lt SelectorSocketTransport closing fd=20 gt Traceback most recent call last File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio selector events py line 918 in write n = self sock send data OSError Errno 9 Bad file descriptor During handling of the above exception another exception occurred Traceback most recent call last File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio sslproto py line 684 in process write backlog self transport write chunk File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio selector events py line 924 in write self fatal error exc Fatal write error on socket transport File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio selector events py line 719 in fatal error self force close exc File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio selector events py line 731 in force close self loop call soon self call connection lost exc File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio base events py line 746 in call soon self check closed File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio base events py line 510 in check closed raise RuntimeError Event loop is closed RuntimeError Event loop is closed ERROR asyncio Fatal error on SSL transport protocol lt asyncio sslproto SSLProtocol object at 0x11dad98e0 gt transport lt SelectorSocketTransport closing fd=17 gt Traceback most recent call last File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio selector events py line 918 in write n = self sock send data OSError Errno 9 Bad file descriptor During handling of the above exception another exception occurred Traceback most recent call last File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio sslproto py line 684 in process write backlog self transport write chunk File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio selector events py line 924 in write self fatal error exc Fatal write error on socket transport File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio selector events py line 719 in fatal error self force close exc File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio selector events py line 731 in force close self loop call soon self call connection lost exc File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio base events py line 746 in call soon self check closed File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 asyncio base events py line 510 in check closed raise RuntimeError Event loop is closed RuntimeError Event loop is closed```

Kevin Kho · Answer

Can I see the Flow code I assume it s just a simple mapped function right

Wilson Bilkovich · Answer

Yeah here s what I m running <https gist github com wilson b28fd38c4619e6056e88ab1aed5d79ef>

Wilson Bilkovich · Answer

```Traceback most recent call last File Users wilson bilkovich pyenv versions 3 9 6 envs addemart lib python3 9 site packages dask kubernetes utils py line 62 in get external address for scheduler service socket getaddrinfo host port File Users wilson bilkovich pyenv versions 3 9 6 lib python3 9 socket py line 953 in getaddrinfo for res in socket getaddrinfo host port family type proto flags socket gaierror Errno 8 nodename nor servname provided or not known``` is a variation of the error I ve also encountered

Wilson Bilkovich · Answer

but all the env variables everywhere seem to be set specifying the hosts and urls

Kevin Kho · Answer

I really can t find any leads on this Just a question though does this persist with the `prefect` image

Wilson Bilkovich · Answer

Let me try

Kevin Kho · Answer

prefect only has a python 3 8 image i think so you may need to downgrade your python env version

Wilson Bilkovich · Answer

Luckily it doesn t get that far I get the same error with `prefecthq prefect latest` with `dask kubernetes` as the sole extra pip package

Kevin Kho · Answer

Yeah I haven t seen this before unfortunately and the searches don t give anything My unsatisfying best advice would be to go down to python 3 8 to maybe find a stable version that way

Wilson Bilkovich · Answer

So I stopped a port forwarding that was going on and it seems to have gone away I went back to using `daskdev dask` to get the python 3 9 support and it worked

Wilson Bilkovich · Answer

It was just me forwarding a port to Kibana though so I don t know how it manages to interfere

Kevin Kho · Answer

I see Portforwarding on the dask side

Wilson Bilkovich · Answer

Actually just some separate port forwarding so I could watch the cluster logs at the same time which is why it puzzles me so much

Kevin Kho · Answer

My god I wonder how you even figured it out sweat smile

Wilson Bilkovich · Answer

Once the port forwarding to Dask failed and I decided to try canceling it could have taken another day to figure out easily haha