Hi everyone. I have a flow that required the `gevent` package. `gevent` will patch the SSL-library a...

Thomas Fredriksen

10/05/2021, 9:26 AM

Hi everyone. I have a flow that required the

gevent

package.

gevent

will patch the SSL-library and requires the following lines to be added before

ssl

is imported in order to avoid an infinite import recursion:

Copy code

import gevent.monkey
gevent.monkey.patch_all()

I am trying to deploy my flow to a Prefect Server instance, but it seems I am not able to monkey-patch early enough, as I am seeing this infinite recursion error in one of my tasks. Is there any way of forcing the server to do this monkey-patch before executing the flow?

Thomas Fredriksen

10/05/2021, 9:28 AM

I should add that I am using the Kubernetes agent, so I suppose that each task must execute the patch before they start?

Kevin Kho

10/05/2021, 2:55 PM

If you need this called before executing a flow, or task, maybe you can do it in a state handler so it gets called before the flow/task is executed?

Thomas Fredriksen

10/05/2021, 5:32 PM

@Kevin Kho - it needs to be called at import-time

Thomas Fredriksen

10/05/2021, 5:34 PM

I am trying to implement a task for Locust (locust.io), but this gevent-problem has halted my progress. The only solution I can see is modifying the prefect CLI to do the monkey-patch before even parsing the arguments, but this kind of defeats the purpose of having a locust-task in prefect in the first place

Thomas Fredriksen

10/05/2021, 5:39 PM

actually found an issue about this: https://github.com/PrefectHQ/prefect/issues/4227

Kevin Kho

10/06/2021, 2:47 PM

You can chime in on the issue and reopen it I think.

4 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.