Hello @Marvin I am seeing this error when running a docker worker behind a company firewall (zScaler). Can you guide me through how to fix this?

File "/opt/anaconda/lib/python3.8/site-packages/httpcore/_backends/anyio.py", line 69, in start_tls
    ssl_stream = await anyio.streams.tls.TLSStream.wrap(
  File "/opt/anaconda/lib/python3.8/site-packages/anyio/streams/tls.py", line 123, in wrap
    await wrapper._call_sslobject_method(ssl_object.do_handshake)
  File "/opt/anaconda/lib/python3.8/site-packages/anyio/streams/tls.py", line 131, in _call_sslobject_method
    result = func(*args)
  File "/opt/anaconda/lib/python3.8/ssl.py", line 944, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError:

It appears your SSL certificate verification is failing while attempting to establish a connection from a Docker worker running behind a company firewall. This is typically because the certificate provided by the server is not trusted by your system, which can be due to the certificate not being signed by a recognized authority, or a missing intermediate certificate in the trust chain. Here's a general approach to resolving SSL certificate issues, although be mindful that your problem might not merely be technical but existential like the many other trials and tribulations life throws at us: 1. Configure SSL Certificates: Ensure that your system or application has the correct SSL certificates installed. You may need to import your company's root certificate or any intermediates to the trusted store of the Docker container. 2. Verify Firewall or Proxy Configuration: Make sure the firewall (zScaler in your case) is configured to allow traffic and has the correct SSL inspection settings. You might need to add exceptions or rules for your Docker worker's traffic. 3. Use the Correct SSL Configuration: Within the Docker container, ensure that the SSL configuration points to the correct certificate files and that the certificates are correctly mounted or accessible to the application. 4. Consult Documentation for Custom SSL Config: If you're dealing with specific proxy or firewall settings, you might need to refer to their documentation for proper SSL setup. 5. GitHub Issue: Since Prefect 2.x is relatively new, you could check for related open issues on GitHub that could shed some light on the problem or report a new one if necessary. Unfortunately, none of the excerpts from the Prefect documentation directly address the SSL issue with Docker workers behind a firewall that you have described. However, you may find some useful information in the links about deployment in different environments: - Developing a new worker type - Daemonize - Push work pools - Legacy Data Processing Addendum Additionally, this GitHub issue seems to discuss difficulties with SSL in a different context and might offer clues or community advice on similar SSL errors. If the problem persists and you are unable to find the answer, consider reaching out to your IT department for help with the firewall settings or to the zScaler community for advice on SSL certificate verification through the firewall.

i have seen a significant amount of ssl issues on linux running python 3.8 in the past, for numerous different reasons (often related to

crypotography

) • are you able to upgrade from 3.8? • sometimes this works

pip install --upgrade certifi

(in your Dockerfile I'd imagine) • other suggestions on S.O.

I will investigate these things, what do you think is the most likely fix as I think a lot of people online mentioned that I could set some environment variables for CA_BUNDLES in the dockerfile?

if you're able to upgrade from 3.8 that would be my first choice, otherwise (sorry its not more helpful) I would just have to start trying things 🤷. as I mentioned, unfortunately I've seen this crop up for many different reasons, but marvin's answer above should be directionally correct