Adam Roderick
03/16/2022, 2:25 PMAnna Geller
03/16/2022, 2:29 PMstored_as_script=True
- see examples here https://github.com/anna-geller/packaging-prefect-flows/tree/master/flows_no_buildKevin Kho
03/16/2022, 2:30 PMbuild=False
Anna Geller
03/16/2022, 2:30 PMif __name__ == "__main__":
docker_storage.add_flow(flow)
flow.register(project_name="community", build=False)
Adam Roderick
03/16/2022, 8:20 PMflow.register()
which builds the image in what seems like a prefect context, because there are several Docker build steps relate to prefect that we didn't define in our Dockerfile. flow.register()
also couples the registration and image building, and this demo app shows how to do registration--but how is the image built?Kevin Kho
03/16/2022, 11:33 PMbuild=False
, you handle image building and uploading. If build=True
, then we upload to the registry for youAnna Geller
03/16/2022, 11:58 PMpip install .
from the root project directory before I register the flow. But you can also move it to the task that needs that import if you don't want to do that. I kind of assumed you have the same dependencies in your local development environment so that you can first run your flow locally before building a Docker image and deploying your flow to production/Prefect Cloud.To use flow script files in Docker, we need the following arguments to be set properly:
- `stored_as_script` = True
- `path` must point to the path in the image
- if this path is different from/opt/prefect/flows/YOUR_FLOW.py,
then you also need to change the default `prefect_directory` to your custom directory
- you need to add the flow to the storage explicitly before registering, e.g.: docker_storage.add_flow(flow)
- finally, registering the flow must happen from Python, since the `prefect register` CLI doesn't have the option
to pass build=False, and this is critical to include to prevent pickling flow and rebuilding the image on the registration
- the parameter `ignore_healthchecks` on the Docker storage is optional and doesn't affect this process at all
example build commands are here https://github.com/anna-geller/packaging-prefect-flows/blob/master/aws_docker_build_commands.bash
LMK if you have any more questions about itAdam Roderick
03/17/2022, 5:33 PM