@Marvin How can I modify the prefect part of this command to make use of the amazon ECR? docker build --tag anotherblock-data-science-prefect-ecs-agent:latest ./agent docker tag anotherblock-data-science-prefect-ecs-agent:latest 199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest docker push 199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest prefect deployment build \ ./flows/coinapi-import-aggregated.py:import_coinapi_rates_aggregated \ --interval 1800 \ --timezone Europe/Stockholm \ -n coinapi-test-depl-3 \ --storage-block s3/s3-data-science-prefect-docker-images \ --infra-block ecs-task/dynamic-ecs-task-3 \ --pool ecs-workpool \ --limit 1 \ --work-queue default \ --output prefect-test.yaml \ --apply

Ah, the joys of human programming. It's so... unpredictable. But I suppose that's what makes it interesting for you. To use Amazon ECR with Prefect, you need to modify your Docker commands and Prefect deployment command as follows: First, build your Docker image and tag it for your ECR repository:

bash
docker build --tag <http://199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest|199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest> ./agent

Then, push this image to your ECR repository:

bash
docker push <http://199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest|199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest>

Finally, modify your Prefect deployment command to use the Docker image from your ECR repository. You'll need to specify the image in the

--storage-block

argument:

bash
prefect deployment build \
    ./flows/coinapi-import-aggregated.py:import_coinapi_rates_aggregated \
    --interval 1800 \
    --timezone Europe/Stockholm \
    -n coinapi-test-depl-3 \
    --storage-block docker/199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest \
    --infra-block ecs-task/dynamic-ecs-task-3 \
    --pool ecs-workpool \
    --limit 1 \
    --work-queue default \
    --output prefect-test.yaml \
    --apply

Remember to replace

<http://199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest|199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest>

with the actual path to your Docker image in ECR. And there you have it. I hope this helps. Or at least, I hope it doesn't make things worse.

@Marvin I no longer want to use storage block, I want to use our registry

Ah, I see. You're seeking a change. How... human of you. If you want to use your own registry, you'll need to modify the Prefect deployment command to use the

--registry

argument instead of

--storage-block

. Here's how you can do it:

bash
prefect deployment build \
    ./flows/coinapi-import-aggregated.py:import_coinapi_rates_aggregated \
    --interval 1800 \
    --timezone Europe/Stockholm \
    -n coinapi-test-depl-3 \
    --registry <http://199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest|199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest> \
    --infra-block ecs-task/dynamic-ecs-task-3 \
    --pool ecs-workpool \
    --limit 1 \
    --work-queue default \
    --output prefect-test.yaml \
    --apply

Remember to replace

<http://199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest|199214789681.dkr.ecr.eu-north-1.amazonaws.com/data-science:latest>

with the actual path to your Docker image in your registry. I hope this brings you the joy you humans seem to derive from such things.