Hello. I have an issue I hope someone can shine some light on. I am using the Docker storage option for Prefect, and trying to set the base_image. The run environment is AWS ECS/FARGATE, and Python 3.8. The goal is to use a custom image stored in AWS ECR:

storage = Docker(registry_url=os.getenv("REGISTRY_URL"),
  image_name=os.getenv("IMAGE_NAME"),
  base_image="<http://nnnnnnnnnnn.dkr.ecr.us-west-2.amazonaws.com/flow-base-image:latest|nnnnnnnnnnn.dkr.ecr.us-west-2.amazonaws.com/flow-base-image:latest>"
  )

and built from a very simple Dockerfile:

FROM prefecthq/prefect:latest-python3.8
ENV PYTHONPATH=$PYTHONPATH:/
COPY ./utilities utilities

Currently, this setup fails. When the flow runs in ECS, it exits immediately with an error: Exit Code 1 Command ["/bin/sh","-c","prefect execute flow-run"] As part of my troubleshooting, I bypassed using the image in ECR and set the base image in the Docker storage for the flow:

storage = Docker(registry_url=os.getenv("REGISTRY_URL"),
  image_name=os.getenv("IMAGE_NAME"),
  base_image="prefecthq/prefect:latest-python3.8"
  )

This too failed with the same error. I reviewed the logs for builds that did not set the base image, and saw this image being used by default: prefecthq/prefect:0.15.4-python3.8 I then used this image in my Docker storage:

storage = Docker(registry_url=os.getenv("REGISTRY_URL"), 
  image_name=os.getenv("IMAGE_NAME"),
  base_image="prefecthq/prefect:0.15.4-python3.8"
  )

This flow runs successfully. Updating my Dockerfile and using that build from ECR also works. So…does anyone have any idea as to why using prefecthq/prefect:latest-python3.8 fails? I would prefer not to pin the version in my Dockerfile.

it's likely an issue that the environment variable for registry URL is available at build time but not at runtime - can you try without env variable?

Hi @Anna Geller. Thanks for your response. Just to clarify, this works:

base_image="prefecthq/prefect:0.15.4-python3.8"

This doesn't work:

base_image="prefecthq/prefect:latest-python3.8"

Error seen in ECS:

Exit Code 1
  Command ["/bin/sh","-c","prefect execute flow-run"]

The environment variable is the same for both, so if it was an issue it would impact both?

thanks for clarifying. This looks like cloudpickle serialization issue. It looks like you are registering your flow from an environment that uses Python 3.8 and Prefect 0.15.4 and that's why using the same base image works. Could you try creating a new virtual environment with the latest Prefect version (with Python >= 3.7) and then try again?