@Marvin using prefect 2.x and a prefect.yaml file I am trying to set up a deployment in a docker container. However I get following error when running deployment: Discovered type 'docker' for work pool 'dev'. Worker 'DockerWorker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923' started! 095413.482 | INFO | prefect.flow_runs.worker - Worker 'DockerWorker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923' submitting flow run '85c1e1db-a989-4a22-a540-8806e017a9d0' 095414.536 | INFO | prefect.worker.docker.dockerworker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923 - Logging into Docker registry... 095415.136 | INFO | prefect.worker.docker.dockerworker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923 - Pulling image ''imagerepo... 095415.846 | INFO | prefect.worker.docker.dockerworker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923 - Creating Docker container 'screeching-platypus'... 095415.989 | INFO | prefect.worker.docker.dockerworker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923 - Docker container 'screeching-platypus' has status 'created' 095416.395 | INFO | prefect.worker.docker.dockerworker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923 - Docker container 'screeching-platypus' has status 'running' 095416.558 | INFO | prefect.flow_runs.worker - Completed submission of flow run '85c1e1db-a989-4a22-a540-8806e017a9d0' frozen runpy128 RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour 095419.430 | ERROR | Flow run 'screeching-platypus' - Flow could not be retrieved from deployment. Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/prefect/engine.py", line 268, in retrieve_flow_then_begin_flow_run flow = await load_flow_from_flow_run(flow_run, client=client)

Your message was too long by 345 tokens - please shorten it and try again. For reference, here's your message at the allowed limit:

using prefect 2.x and a prefect.yaml file I am trying to set up a deployment in a docker container. However I get following error when running deployment:

Discovered type 'docker' for work pool 'dev'. Worker 'DockerWorker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923' started! 095413.482 | INFO | prefect.flow_runs.worker - Worker 'DockerWorker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923' submitting flow run '85c1e1db-a989-4a22-a540-8806e017a9d0' 095414.536 | INFO | prefect.worker.docker.dockerworker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923 - Logging into Docker registry... 095415.136 | INFO | prefect.worker.docker.dockerworker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923 - Pulling image 'geomatic.azurecr.io/analyticsfoundation:3.0'... 095415.846 | INFO | prefect.worker.docker.dockerworker 2

@Marvin using prefect 2.x and a prefect.yaml file I am trying to set up a deployment in a docker container. However I get following error when running deployment: 095416.395 | INFO | prefect.worker.docker.dockerworker 2bd1380a-a3ed-49b1-9dd9-d3e0dc7a4923 - Docker container 'screeching-platypus' has status 'running' 095416.558 | INFO | prefect.flow_runs.worker - Completed submission of flow run '85c1e1db-a989-4a22-a540-8806e017a9d0' <frozen runpy>128 RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour 095419.430 | ERROR | Flow run 'screeching-platypus' - Flow could not be retrieved from deployment. Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/prefect/engine.py", line 268, in retrieve_flow_then_begin_flow_run flow = await load_flow_from_flow_run(flow_run, client=client)

It seems you are encountering an issue where the flow cannot be retrieved from the deployment when running it in a Docker container in Prefect 2.x. Here are some findings that might help you resolve this issue: ### Potential Issues and Solutions: 1. Kubernetes Specific Deployment Issues: - Issue #10864: Sometimes the custom image specified isn't used, and the standard Prefect image is used instead. This can occur when KubernetesJob doesn't apply settings properly. You may need to use

infra_overrides

as a workaround to ensure that the correct image is used. - Source: GitHub Issue #10864 2. Environment Setup Issues: - Ensure that your

prefect.yaml

is correctly specifying the entrypoint, Dockerfile, and any platforms if you're using Docker. Missing or misconfigured paths can cause the flow retrieval to fail. 3. Git Cloning Issues: - Issue #10420: When re-running deployments that clone a repository, you may encounter issues if the directory already exists. Make sure your deployment is handling the teardown and setup properly to avoid this. - Source: GitHub Issue #10420 ### Steps for Debugging: 1. Prefect Deployment Configuration: - Ensure that your

prefect.yaml

is correctly configured. Here is an example:

yaml
   # Generic metadata about this project
   name: my_project
   prefect-version: 2.11.0

   # build section allows you to manage and build docker images
   build:
     - prefect.deployments.steps.run_shell_script:
         id: build-flows
         script: docker build -t my-container .

   # pull section allows you to provide instructions for cloning this project in remote locations
   pull:
     - prefect.deployments.steps.set_working_directory:
         directory: /opt/prefect/flows

   # the deployments section allows you to provide configuration for deploying flows
   deployments:
     - name: my-deployment
       tags: []
       schedule: null
       entrypoint: "flows/my_flow.py:my_flow"
       work_pool:
         name: "docker-container"
         job_variables:
           image: "my-container"

2. Check your Dockerfile: - Make sure that the Dockerfile used for the deployment includes all necessary dependencies and correctly sets up the directory structure:

dockerfile
   FROM python:3.11-slim
   COPY requirements.txt /tmp/
   RUN pip install -r /tmp/requirements.txt
   COPY . /opt/prefect
   WORKDIR /opt/prefect

3. Logging: - Add detailed logging to better understand where it might be failing. This could be useful in identifying issues with loading the flow. If these steps don't resolve the issue, consider searching the Prefect Documentation for specific keywords that match your environment setup or deployment config. Additionally, visiting Prefect's GitHub Issues might provide insights on similar issues faced by other users.