Getting this error when I try to use Docker storage

ValueError: Your docker image failed to build!  Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.

Does

python_dependencies

take care of the

pip install

for me? Or do I have to provide a custom Dockerfile that does the

COPY requirements.txt

and

RUN pip install -r requirements.txt

?

@Joseph Haaga

python_dependencies

should have taken care of that for you

It’s also possible the dependency that you’re missing is your own module that the flow lives in.

print(f"Building Docker image '{flow.name}'")
    flow.storage = Docker(
        registry_url="<http://REDACTED.ecr.us-east-1.amazonaws.com|REDACTED.ecr.us-east-1.amazonaws.com>",
        image_name=flow.name,
        python_dependencies=reqs,
    )

Most of the trace was

pip install

, but here is the rest

Imported Flow: 'Extract Entities from CNBC'
Building Docker image '{flow.name}'
Setting Execution Environment
Registering Flow 'Extract Entities from CNBC' to Project Relationship Discovery
/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/core/flow.py:1623: UserWarning: No result handler was specified on your Flow. Cloud features such as input caching and resuming task runs from failure may not work properly.
  registered_flow = client.register(

[2020-10-30 21:08:08] INFO - prefect.Docker | Building the flow's Docker storage...
Traceback (most recent call last):
  File "/Users/josephhaaga/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/josephhaaga/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect_deployer/__main__.py", line 5, in <module>
    exit(main())
  File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect_deployer/main.py", line 27, in main
    print(f"Installing Flow requirements")
  File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/core/flow.py", line 1623, in register
    registered_flow = client.register(
  File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/client/client.py", line 734, in register
    serialized_flow = flow.serialize(build=build)  # type: Any
  File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/core/flow.py", line 1458, in serialize
    storage = self.storage.build()  # type: Optional[Storage]
  File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/environments/storage/docker.py", line 351, in build
    self._build_image(push=push)
  File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/environments/storage/docker.py", line 417, in _build_image
    raise ValueError(
ValueError: Your docker image failed to build!  Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.

@Joseph Haaga silly question, but are you sure you’ve specified all of your dependencies? Sometimes python deps require others (pandas requires psycopg2 and sqlalchemy if I use

read_sql

with postgres)

hmm I’ll double check but I believe I am; I froze my env to a

requirements.txt

and am parsing that to pass as the

python_dependencies

It may also be worth looking at the discussion at https://prefect-community.slack.com/archives/CL09KU1K7/p1603476596033800

If you are, maybe double-check that the parsing is going as the storage object is expecting?

Although it's erroring in a different way so my apologies if that's a misleading direction.

Still getting this error on a very simple flow that only depends on

prefect.Flow

and

prefect.task

(the hello-world one). Is the typical deployment pattern to supply a base Dockerfile that COPY’s the source code into the image? I figured whatever

_build_image

occurs on

flow.register

would handle putting the source into the Docker image, but I’m becoming skeptical

Can you please share a full reproducible case with your code? (a redacted registry url is fine)