Hello, im quite confused on how deployment works w...
# ask-community
m
Hello, im quite confused on how deployment works with docker. What im trying to do is to have a custom image that i want my flow to be run in In my mind the prefect should pull that image and execute my flows inside the image i have defined, but i get ModuleNotFoundError: No module named 'prefect_docker And i dont understand where the worker is executing this (inside the docker?). I created my pool with the type docker:
Copy code
prefect work-pool create --type docker cccg-pool
Here is the yaml:
name: prefect
prefect-version: 2.11.0
# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect_docker.deployments.steps.pull_docker_image:
image_name:
self-hosted.com:1.0.0.dev3
deployments:
- name: cccg
version: 1.0.0
tags: []
concurrency_limit: 1
description: Camera Constant Correction Generator deployment
parameters: {}
work_pool:
name: cccg-pool
work_queue_name: default
job_variables: {}
job_variables: {}
enforce_parameter_schema: true
schedules: []
entrypoint: scripts/main.py:cam_flow
Output from worker:
└─[$] <> prefect worker start --pool "cccg-pool"
Discovered type 'docker' for work pool 'cccg-pool'. Worker 'DockerWorker 61595df4-9d66-4fbb-8627-6768f561f611' started! 155243.783 | INFO | prefect.flow_runs.worker - Worker 'DockerWorker 61595df4-9d66-4fbb-8627-6768f561f611' submitting flow run '9665a4e6-dcf4-46d4-ba32-9dcf6d5ebaed' 155245.663 | INFO | prefect.worker.docker.dockerworker 61595df4-9d66-4fbb-8627-6768f561f611 - Creating Docker container 'onyx-worm'... 155245.682 | INFO | prefect.worker.docker.dockerworker 61595df4-9d66-4fbb-8627-6768f561f611 - Docker container 'onyx-worm' has status 'created' 155245.834 | INFO | prefect.worker.docker.dockerworker 61595df4-9d66-4fbb-8627-6768f561f611 - Docker container 'onyx-worm' has status 'running' 155245.890 | INFO | prefect.flow_runs.worker - Completed submission of flow run '9665a4e6-dcf4-46d4-ba32-9dcf6d5ebaed' 125247.003 | INFO | prefect.flow_runs.runner - Opening process... 125248.157 | ERROR | prefect.engine - Engine execution of flow run '9665a4e6-dcf4-46d4-ba32-9dcf6d5ebaed' exited with unexpected exception Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 161, in run_steps step_output = await run_step(step, upstream_outputs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 131, in run_step step_func = _get_function_for_step(fqn, requires=keywords.get("requires")) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 76, in _get_function_for_step step_func = import_object(fully_qualified_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/utilities/importtools.py", line 216, in import_object module = load_module(module_name) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/utilities/importtools.py", line 187, in load_module return importlib.import_module(module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "frozen importlib._bootstrap", line 1387, in _gcd_import File "frozen importlib._bootstrap", line 1360, in _find_and_load File "frozen importlib._bootstrap", line 1310, in _find_and_load_unlocked File "frozen importlib._bootstrap", line 488, in _call_with_frames_removed File "frozen importlib._bootstrap", line 1387, in _gcd_import File "frozen importlib._bootstrap", line 1360, in _find_and_load File "frozen importlib._bootstrap", line 1310, in _find_and_load_unlocked File "frozen importlib._bootstrap", line 488, in _call_with_frames_removed File "frozen importlib._bootstrap", line 1387, in _gcd_import File "frozen importlib._bootstrap", line 1360, in _find_and_load File "frozen importlib._bootstrap", line 1324, in _find_and_load_unlocked ModuleNotFoundError: No module named 'prefect_docker'