Hi - I am being lazy about re-configuring another docker build push interface. Is it possible to use

prefect.environments.storage.docker

as the mechanism to build and push containers while using Non-Docker Storage for Containerized Environments? (I ran into a bug deploying this, and wondering if this might be a reason)

flow.storage = S3(
    bucket="s3-prefect-flow-storage",
    secrets=["AWS_CREDENTIALS"],
    )

docker = Docker(
    registry_url=ecr_repo_url,
    python_dependencies=[
        "pandas",...],
    dockerfile=docker_flpth,
    image_name="annoying_docker",
    image_tag="latest",
    local_image=True
    )
docker.build(push=True)

Hi @itay livni personally I think the image you set in the

environment.metadata

should be built independently using something like Docker’s CLI. The

Docker

storage has a lot of logic baked into it around flow storage and therefore does some extra things for storing the flow that you will not need in your metadata image

Although I do like the health checks. It's like a pre-push hook.

Yeah they are nice 🙂 you could call them directly https://github.com/PrefectHQ/prefect/blob/master/src/prefect/environments/storage/_healthcheck.py although some might not be viable without being inside your flow’s image

@josh 1. I built and pushed the docker image to ecr using the aws cli (docker-gist) 2. Ran the deploy script (gist) 3. Started the `FargateAgent (start-agent-gist) successfully. 4. Invoked a quick run The flow got scheduled but did not run. No ECS tasks began and the agent doesn't seem to know that a quick run was invoked. -- Thanks

The default label is

s3-flow-storage

@josh Thanks. That worked. The new error from the Cloud is

Failed to load and execute Flow's environment: ValueError('Flow is not contained in this Storage')

I changed storage and agent configuration to

labels=s3-flow-storage

.

How are you registering the flow?

@josh

pushlog = etl_moc_flow.register(
    project_name="market-on-close", 
    build=True,
    set_schedule_active=False
    )

Hmm interesting, and this flow has both s3 storage and an image name in the environment metadata? FWIW I just did this and it was working for me 🙂

@josh the enviroment is simply configured like this;

etl_moc_flow.environment = LocalEnvironment(
    metadata={"image": image}
    )

Is there anything else I have to add?

No that should be it! Would you mind opening an issue so we can track?

Will do. Thanks.