Joseph Haaga
10/30/2020, 9:11 PMValueError: Your docker image failed to build! Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.
Does python_dependencies
take care of the pip install
for me? Or do I have to provide a custom Dockerfile that does the COPY requirements.txt
and RUN pip install -r requirements.txt
?Dylan
python_dependencies
should have taken care of that for youZanie
Joseph Haaga
10/30/2020, 9:20 PMprint(f"Building Docker image '{flow.name}'")
flow.storage = Docker(
registry_url="<http://REDACTED.ecr.us-east-1.amazonaws.com|REDACTED.ecr.us-east-1.amazonaws.com>",
image_name=flow.name,
python_dependencies=reqs,
)
Most of the trace was pip install
, but here is the rest
Imported Flow: 'Extract Entities from CNBC'
Building Docker image '{flow.name}'
Setting Execution Environment
Registering Flow 'Extract Entities from CNBC' to Project Relationship Discovery
/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/core/flow.py:1623: UserWarning: No result handler was specified on your Flow. Cloud features such as input caching and resuming task runs from failure may not work properly.
registered_flow = client.register(
[2020-10-30 21:08:08] INFO - prefect.Docker | Building the flow's Docker storage...
Traceback (most recent call last):
File "/Users/josephhaaga/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/Users/josephhaaga/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect_deployer/__main__.py", line 5, in <module>
exit(main())
File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect_deployer/main.py", line 27, in main
print(f"Installing Flow requirements")
File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/core/flow.py", line 1623, in register
registered_flow = client.register(
File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/client/client.py", line 734, in register
serialized_flow = flow.serialize(build=build) # type: Any
File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/core/flow.py", line 1458, in serialize
storage = self.storage.build() # type: Optional[Storage]
File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/environments/storage/docker.py", line 351, in build
self._build_image(push=push)
File "/Users/josephhaaga/Documents/Code/ETL/prefect-deployer/venv/lib/python3.8/site-packages/prefect/environments/storage/docker.py", line 417, in _build_image
raise ValueError(
ValueError: Your docker image failed to build! Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.
Dylan
read_sql
with postgres)Joseph Haaga
10/30/2020, 9:45 PMrequirements.txt
and am parsing that to pass as the python_dependencies
Zanie
Dylan
Zanie
Joseph Haaga
10/30/2020, 10:29 PMprefect.Flow
and prefect.task
(the hello-world one).
Is the typical deployment pattern to supply a base Dockerfile that COPY’s the source code into the image? I figured whatever _build_image
occurs on flow.register
would handle putting the source into the Docker image, but I’m becoming skepticalZanie