Thread
#best-practices-coordination-plane
    Aaron Goebel

    Aaron Goebel

    3 months ago
    Some of my prefect tasks need to use a database connection. Connections are not serializable, so tasks would have to construct one on run, but this is pricy. If you have a bunch of tasks that require the connection is there a way to provide them with something like a connection pool?
    Kevin Kho

    Kevin Kho

    3 months ago
    Are you using Dask? If you use Dask, then you absolutely need to serialize because that’s how Dask passes data. If you aren’t using Dask, you can try turning off the serialization:
    @task(checkpoint=False)
    def get_client():
        ....
    Aaron Goebel

    Aaron Goebel

    3 months ago
    Oh great. These tasks don't need dask, but the results a couple tasks down are executed on dask