A few things could cause this with a Docker work pool, but the most common issue is: the object/class you want to create exists in your local environment, but not inside the container that the worker starts. With Docker work pools, your flow runs inside the image/container, so anything created at runtime must be available there: - your Python package/module - any Prefect integration package like

prefect-aws

,

prefect-gcp

, etc. - environment variables / credentials - writable filesystem path, if you're creating files locally A few common failure modes: 1. Missing dependencies in the container If your flow does something like:

python
from prefect_aws.s3 import S3Bucket
bucket = S3Bucket.load("my-bucket")

then

prefect-aws

must be installed in the Docker image. If it’s only installed on your laptop, it will fail in the worker container. 2. Custom classes/modules are not in the image If you create your own objects from local code, that code must be copied into the image or pulled from source at runtime. 3. You’re creating files/objects on local disk and expecting them to persist Docker worker containers are ephemeral. Files created during one run usually won’t be there for the next run. 4. Import/load pattern issues with Blocks For Prefect Blocks, the class must be importable in the container before loading. For example:

python
from prefect_aws.s3 import S3Bucket

bucket = S3Bucket.load("my-bucket")

not just a generic load if the block class package is missing. To help you concretely, can you share: * the exact error message / traceback * what kind of “objects” you’re trying to create: - Python class instances? - Prefect Blocks? - files/directories? - cloud clients like boto3 / db connections? * how you built/deployed the flow: - custom Dockerfile? -

flow.deploy(...)

with an image? -

prefect deploy

? If you want, paste:

your Dockerfile
your deployment config
the failing code snippet
the traceback

and I can pinpoint the issue.

That should work in principle, so the likely problem is how/where the Kafka producer is being created. With a Docker work pool, the safest pattern is: create the Kafka producer inside the task/function that uses it, and do not pass or capture it from outside. Example:

python
from prefect import task, flow

@task
def send_message():
    from kafka import KafkaProducer

    producer = KafkaProducer(
        bootstrap_servers=["kafka:9092"]
    )
    producer.send("my-topic", b"hello")
    producer.flush()
    producer.close()

@flow
def my_flow():
    send_message()

if __name__ == "__main__":
    my_flow()

A few important gotchas: 1. Do not create the producer globally

python
producer = KafkaProducer(...)  # avoid this

@task
def send():
    producer.send(...)

That often breaks because the object holds network/socket state and is not safe to reuse across task/process boundaries. 2. Make sure the Kafka library is installed in the image Your Docker image must include the client package, e.g.

kafka-python

or

confluent-kafka

. For example in your Dockerfile:

dockerfile
FROM prefecthq/prefect:3-latest
RUN pip install kafka-python

3. Make sure the broker hostname is reachable from inside the container

localhost:9092

usually means “this same container”, not your Kafka broker. If Kafka is another container, use its container/service name, for example:

python
bootstrap_servers=["kafka:9092"]

not:

python
bootstrap_servers=["localhost:9092"]

4. Pass config, not client objects Good:

python
@task
def send_message(bootstrap_servers, topic, payload):
    from kafka import KafkaProducer
    producer = KafkaProducer(bootstrap_servers=bootstrap_servers)
    producer.send(topic, payload)
    producer.flush()
    producer.close()

Bad:

python
producer = KafkaProducer(...)

@task
def send_message(producer, topic, payload):
    ...

To narrow this down, please paste:

the exact error traceback
the task/function code where you create the producer
the bootstrap_servers value you are using

My first suspicion is one of these three: - the Kafka package is missing in the container -

localhost

is wrong from inside Docker - the producer is being created outside the task/function and reused If helpful, I can also help you rewrite your Kafka task in a Docker-safe Prefect pattern.