Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi Guys, Is there a way to supply MySQL connection to multiple tasks? Currently, I am getting this error :
"TypeError: can't pickle _mysql_connector.MySQL objects"

Afaik the best practice is to pass connection information like dbname, host, user, pwd directly to teh MySQLTask.

Passing the connector would cause Prefect to not being able to serialize your flow, hence the pickling issue

I tried importing MySQLFetch using: prefect.tasks.mysql.mysql.MySQLFetch, but it is saying no such module exists. I have installed prefect  with all_extras.

Strange.
If you can share the code that is not working maybe someone from the community can help you figuring it out

That module was moved in a recent prefect release. You can try upgrading, or importing from `prefect.contrib....`

And yeah, as <@U019RALEAUW> said, this has to do with connection objects not being pickleable. We recommend creating a new connection in each task. There's some ongoing work in core to make this more efficent, so tasks running in the same process can share a connection between them, but that work hasn't been completed yet.

I have created new connections in each of the tasks and it worked. Thanks <@U011EKN35PT> and <@U019RALEAUW>