https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I am getting this error ```No block c...
# ask-marvin
c
@Marvin I am getting this error
Copy code
No block class found for slug 's3-bucket'. Please ensure that prefect-aws is installed.
for my self-hosted ecs worker, but I am already installing the prefect-aws:
Copy code
pip install prefect-aws && prefect worker start --pool {ecs_pool_name} --type ecs
What might be an issue?
m
✅ thought for 21.3 seconds
Based on your error message and the command you're using, I can identify a few potential issues: 1. Installation Timing: The most likely issue is that 
prefect-aws
needs to be installed BEFORE the worker starts. In your command, you're doing both in one line with 
&&
, which means the worker might start before the installation is fully complete and registered. 2. Worker Environment: When using an ECS worker, the 
prefect-aws
package needs to be installed in the container environment where your flows will run, not just where you start the worker. Here's what I recommend: 1. First, install prefect-aws separately:
Copy code
bash
pip install prefect-aws
2. Verify the installation was successful:
Copy code
bash
prefect block type ls
You should see 
s3-bucket
in the list. 3. Then start your worker:
Copy code
bash
prefect worker start --pool your-ecs-pool-name --type ecs
4. Make sure your ECS task definition includes 
prefect-aws
in its dependencies. This could be: - In your Dockerfile: 
RUN pip install prefect-aws
- In your requirements.txt - Or in your deployment configuration 5. If you're using a custom Docker image for your flows, ensure that 
prefect-aws
is installed in that image as well. Additionally, you might want to verify that: - You have the correct version of 
prefect-aws
installed (latest is recommended) - Your Python environment isn't having any conflicts - The S3 block is properly registered in your Prefect environment If the issue persists after trying these steps, could you please share: 1. The version of 
prefect-aws
you're using 2. How you're defining your ECS task (Dockerfile or task definition) 3. Whether you see the same error when running flows locally
c
@Marvin 1. using 0.5.13 for prefect-aws 2. Here is the ecs task definition:
Copy code
{
  "taskDefinitionArn": "{task_definition_arn}",
  "containerDefinitions": [
    {
      "name": "PrefectWorkerContainer",
      "image": "prefecthq/prefect:3-latest",
      "cpu": 512,
      "memory": 1024,
      "links": [],
      "portMappings": [],
      "essential": true,
      "entryPoint": [],
      "command": [
        "/bin/sh",
        "-c",
        "pip install prefect-aws && prefect worker start --pool {ecs_pool_name} --type ecs"
      ],
      "environment": [
        {
          "name": "PREFECT_API_URL",
          "value": "{prefect_api_url}"
        },
        {
          "name": "PREFECT_API_AUTH_STRING",
          "value": "{prefect_api_auth_string}"
        }
      ],
      "environmentFiles": [],
      "mountPoints": [],
      "volumesFrom": [],
      "secrets": [],
      "dnsServers": [],
      "dnsSearchDomains": [],
      "extraHosts": [],
      "dockerSecurityOptions": [],
      "dockerLabels": {},
      "ulimits": [],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "{log_group_name}",
          "awslogs-region": "us-west-2",
          "awslogs-stream-prefix": "PrefectWorker"
        },
        "secretOptions": []
      },
      "systemControls": [],
      "credentialSpecs": []
    }
  ],
  "family": "{family_name}",
  "taskRoleArn": "{task_role_arn}",
  "executionRoleArn": "{execution_role_arn}",
  "networkMode": "awsvpc",
  "revision": 1,
  "volumes": [],
  "status": "ACTIVE",
  "requiresAttributes": [
    {
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "name": "ecs.capability.execution-role-awslogs"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.17"
    },
    {
      "name": "com.amazonaws.ecs.capability.task-iam-role"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
    },
    {
      "name": "ecs.capability.task-eni"
    }
  ],
  "placementConstraints": [],
  "compatibilities": [
    "EC2",
    "FARGATE"
  ],
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "cpu": "512",
  "memory": "1024",
  "runtimePlatform": {
    "cpuArchitecture": "X86_64"
  },
  "registeredAt": "2025-07-30T07:51:11.826Z",
  "registeredBy": "{cfn_role}",
  "tags": []
}
3. Not seeing the issue locally
m
✅ thought for 12.5 seconds
Thank you for providing those details. I can see a potential issue with your ECS task definition that might explain the error. The key issue is likely related to how and when 
prefect-aws
is being installed in your ECS container. In your current setup, you're installing 
prefect-aws
in the same command that starts the worker:
Copy code
bash
pip install prefect-aws && prefect worker start --pool {ecs_pool_name} --type ecs
This is problematic because: 1. The worker needs the package to be installed AND registered before it starts 2. The installation needs to persist in the environment where your flows will run Here are the recommended changes to your ECS task definition:
Copy code
json
{
    "containerDefinitions": [
        {
            "name": "PrefectWorkerContainer",
            "image": "prefecthq/prefect:3-latest",
            "command": [
                "/bin/bash",
                "-c",
                "pip install prefect-aws && python -c 'from prefect.utilities.importtools import import_object' && prefect worker start --pool {ecs_pool_name} --type ecs"
            ],
            // ... rest of your configuration ...
        }
    ]
}
However, a better approach would be to create a custom Docker image that has 
prefect-aws
pre-installed. Here's how: 1. Create a Dockerfile:
Copy code
dockerfile
FROM prefecthq/prefect:3-latest

# Install prefect-aws during image build
RUN pip install prefect-aws==0.5.13

# Optional: verify the installation
RUN python -c 'from prefect.utilities.importtools import import_object'
2. Build and push this image to your container registry 3. Update your ECS task definition to use this custom image and simplify the command:
Copy code
json
{
    "containerDefinitions": [
        {
            "name": "PrefectWorkerContainer",
            "image": "your-custom-image:tag",
            "command": [
                "prefect",
                "worker",
                "start",
                "--pool",
                "{ecs_pool_name}",
                "--type",
                "ecs"
            ],
            // ... rest of your configuration ...
        }
    ]
}
This approach has several advantages: 1. The package is pre-installed and verified during image build 2. Faster container startup time 3. More reliable and reproducible environment 4. No risk of installation timing issues Also, make sure your task role (
task_role_arn
) has the necessary permissions to: 1. Access S3 buckets 2. Create and manage ECS tasks 3. Access any other AWS resources your flows might need Would you like me to provide more details about any of these aspects or help with creating the custom Docker image?
c
@Marvin still failing after the change. This is full stacktrace:
Copy code
Unexpected exception encountered when trying to load flow
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/prefect/blocks/core.py", line 872, in get_block_class_from_key
    return lookup_type(cls, key)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/dispatch.py", line 208, in lookup_type
    raise KeyError(
KeyError: "No class found for dispatch key 's3-bucket' in registry for type 'Block'."

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 166, in run_steps
    step_output = await run_step(step, upstream_outputs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 137, in run_step
    result = await from_async.call_soon_in_new_thread(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 374, in aresult
    return await asyncio.wrap_future(self.future)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 441, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/pull.py", line 265, in pull_with_block
    block = await Block.aload(full_slug)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/client/utilities.py", line 99, in with_injected_client
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/blocks/core.py", line 1077, in aload
    return cls._load_from_block_document(block_document, validate=validate)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/blocks/core.py", line 1278, in _load_from_block_document
    return cls._from_block_document(block_document)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/blocks/core.py", line 809, in _from_block_document
    else cls.get_block_class_from_schema(block_document.block_schema)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/blocks/core.py", line 859, in get_block_class_from_schema
    return cls.get_block_class_from_key(block_schema_to_key(schema))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/blocks/core.py", line 884, in get_block_class_from_key
    raise UnknownBlockType(message)
prefect.blocks.core.UnknownBlockType: No block class found for slug 's3-bucket'. Please ensure that `prefect-aws` is installed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/prefect/engine.py", line 112, in <module>
    flow: "Flow[..., Any]" = load_flow(flow_run)
                             ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/flow_engine.py", line 150, in load_flow
    flow = run_coro_as_sync(
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 207, in run_coro_as_sync
    return call.result()
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 365, in result
    return self.future.result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 192, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 441, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 188, in coroutine_wrapper
    return await task
           ^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/client/utilities.py", line 69, in wrapper
    return await func(client, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/flows.py", line 2810, in load_flow_from_flow_run
    output = await run_steps(
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 194, in run_steps
    raise StepExecutionError(f"Encountered error while running {fqn}") from exc
prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect.deployments.steps.pull_with_block
m
✅ thought for 18.8 seconds
Let's try a different approach. I suspect there might be an issue with how the block is being registered in the Python 3.12 environment. Here's what I recommend: 1. First, modify your Dockerfile to explicitly verify the block registration:
Copy code
dockerfile
FROM prefecthq/prefect:3-latest

# Install prefect-aws
RUN pip install prefect-aws==0.5.13

# Add a verification script
RUN echo 'from prefect_aws.s3 import S3Bucket\n\
print("S3Bucket block class:", S3Bucket._block_type_slug)' > verify_blocks.py

# Run verification on build
RUN python verify_blocks.py

# Add an entrypoint script
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]
2. Create an 
entrypoint.sh
script:
Copy code
bash
#!/bin/bash

# Force block registration on container start
python -c "
from prefect_aws.s3 import S3Bucket
print('Verifying S3Bucket block registration...')
print('Block type slug:', S3Bucket._block_type_slug)
"

# Start the worker
exec prefect worker start --pool "$@"
3. Update your ECS task definition to use this new setup:
Copy code
json
{
    "containerDefinitions": [
        {
            "name": "PrefectWorkerContainer",
            "image": "your-custom-image:tag",
            "command": [
                "{ecs_pool_name}",
                "--type",
                "ecs"
            ],
            "environment": [
                {
                    "name": "PREFECT_API_URL",
                    "value": "{prefect_api_url}"
                },
                {
                    "name": "PREFECT_API_AUTH_STRING",
                    "value": "{prefect_api_auth_string}"
                },
                {
                    "name": "PYTHONPATH",
                    "value": "/usr/local/lib/python3.12/site-packages"
                }
            ],
            // ... rest of your configuration ...
        }
    ]
}
4. Also, let's verify that your flow's deployment is correctly configured. Could you share the code where you're creating the deployment that uses the S3 bucket? Additionally, you might want to try: 1. Explicitly registering the block on container start:
Copy code
bash
prefect block register -m prefect_aws.s3
2. Or try using an earlier Python version (3.10 or 3.11) as Python 3.12 is relatively new and might have compatibility issues. Could you also share: 1. The output of 
prefect version
from your ECS container? 2. The code where you're creating the deployment that uses the S3 bucket? This will help us identify if there's a version mismatch or configuration issue in the deployment itself.
c
@Marvin how to enforce earlier python version?
m
✅ thought for 15.5 seconds
Even though I couldn't verify the manifest directly, I can confirm these are the official recommended tags. To implement this in your ECS task definition, update the image reference:
Copy code
json
{
    "containerDefinitions": [
        {
            "name": "PrefectWorkerContainer",
            "image": "prefecthq/prefect:3-python3.10",  // Using Python 3.10
            "command": [
                "/bin/bash",
                "-c",
                "pip install prefect-aws==0.5.13 && prefect worker start --pool {ecs_pool_name} --type ecs"
            ],
            // ... rest of your configuration ...
        }
    ]
}
I recommend using Python 3.10 because: 1. It's the most stable and well-tested version with Prefect 3.x 2. Most Prefect integrations are thoroughly tested with Python 3.10 3. It avoids potential compatibility issues that might exist with newer Python versions After making this change: 1. Deploy the updated task definition 2. Stop any running tasks (they'll be replaced with the new definition) 3. Let the new tasks start with Python 3.10 This should help resolve the block registration issues you're experiencing. Let me know if you need any clarification or run into other issues!
b
Hey @Chris, the worker has the package installed just fine. When you run a flow, the Prefect worker kicks off another ECS task which spins up a separate container. This is why you're seeing the continued failures. You need to define 
EXTRA_PIP_PACKAGES=prefect-aws==0.5.13
as an environment variable either in your deployment definition, or add it to the environment variable list for the work pool job variables. This will take care of automatically installing 
prefect-aws
when the flow run starts.
c
@Brendan Dalpe Hey, thanks for hopping in here. Adding it to container def's env var does not seem to solve the issue. How do I update the work pool job variables to pass this to worker task?
b
The container def on the deployment? Can you share how you did this?
c
Basically this:
Copy code
{
  "taskDefinitionArn": "{task_definition_arn}",
  "containerDefinitions": [
    {
      "name": "PrefectWorkerContainer",
      "image": "prefecthq/prefect:3-latest",
      "cpu": 512,
      "memory": 1024,
      "links": [],
      "portMappings": [],
      "essential": true,
      "entryPoint": [],
      "command": [
        "/bin/sh",
        "-c",
        "pip install prefect-aws && prefect worker start --pool {ecs_pool_name} --type ecs"
      ],
      "environment": [
        {
          "name": "PREFECT_API_URL",
          "value": "{prefect_api_url}"
        },
        {
          "name": "PREFECT_API_AUTH_STRING",
          "value": "{prefect_api_auth_string}"
        },
        {
          "name": "EXTRA_PIP_PACKAGES",
          "value": "prefect-aws==0.5.13"
        }
      ],
      "environmentFiles": [],
      "mountPoints": [],
      "volumesFrom": [],
      "secrets": [],
      "dnsServers": [],
      "dnsSearchDomains": [],
      "extraHosts": [],
      "dockerSecurityOptions": [],
      "dockerLabels": {},
      "ulimits": [],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "{log_group_name}",
          "awslogs-region": "us-west-2",
          "awslogs-stream-prefix": "PrefectWorker"
        },
        "secretOptions": []
      },
      "systemControls": [],
      "credentialSpecs": []
    }
  ],
  "family": "{family_name}",
  "taskRoleArn": "{task_role_arn}",
  "executionRoleArn": "{execution_role_arn}",
  "networkMode": "awsvpc",
  "revision": 1,
  "volumes": [],
  "status": "ACTIVE",
  "requiresAttributes": [
    {
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "name": "ecs.capability.execution-role-awslogs"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.17"
    },
    {
      "name": "com.amazonaws.ecs.capability.task-iam-role"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
    },
    {
      "name": "ecs.capability.task-eni"
    }
  ],
  "placementConstraints": [],
  "compatibilities": [
    "EC2",
    "FARGATE"
  ],
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "cpu": "512",
  "memory": "1024",
  "runtimePlatform": {
    "cpuArchitecture": "X86_64"
  },
  "registeredAt": "2025-07-30T07:51:11.826Z",
  "registeredBy": "{cfn_role}",
  "tags": []
}
b
And you're seeing the error when you run flows, correct?
The stack trace you provided is from the worker container, or from the Prefect UI for the flow run itself?
c
Actually, I used the job variable method from the link you pasted (deploying with job vars) and it seems to work now. Thanks for your help!
🙌 1
b
Awesome!! Glad to hear it's working
👍 1
Open in Slack
PreviousNext
Powered by