https://prefect.io
Join Slack
I ran into an issue with the docker deployment, my image eventually built fine and I deployed it to ...
# pacc-london-2023
s
I ran into an issue with the docker deployment, my image eventually built fine and I deployed it to the prefect cloud fine, had worker pool setup and worker running but when I tried to run it from prefect cloud it got an error that the working directory /opt/prefect/prefect_training didn't exist.
My prefect.yaml:
Copy code
# Welcome to your prefect.yaml file! You can you this file for storing and managing
# configuration for deploying your flows. We recommend committing this file to source
# control along with your flow code.

# Generic metadata about this project
name: prefect_training
prefect-version: 2.10.15

# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
    requires: prefect-docker>=0.3.0
    image_name: get-the-weather
    tag: latest
    dockerfile: auto
    push: false

# push section allows you to manage if and how this project is uploaded to remote locations
push:

# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect/prefect_training

# the deployments section allows you to provide configuration for deploying flows
deployments:
- name:
  version:
  tags: []
  description:
  schedule: {}
  flow_name:
  entrypoint:
  parameters: {}
  work_pool:
    name:
    work_queue_name:
    job_variables:
      image: '{{ build_image.image }}'
- name: weather-docker
  version:
  tags: []
  description:
  entrypoint: 101.py:run_pipe
  parameters: {}
  work_pool:
    name: docker-work
    work_queue_name:
    job_variables: {}
  schedule:
a
I have the same error too, with directory 
/opt/prefect/{mydirectory}
a
Same
j
try just leaving 
/opt/prefect
?
a
That isn’t working either
😭 1
Jeff just helped me fix it
a few things: 1. make sure you’re on prefect 2.10.16 (we were on 15 yesterday) but this probably isn’t affecting anything, just a fyi 2. make sure you’ve got a docker worker pool not just a plain worker pool. 
prefect worker start -t docker -p docker-pool
3. (typing)
3. 
push: false
in the YAML
4. the deployment config in the YAML is kind of confusing. when you do prefect init and pick docker, it creates an empty deploy config like this:
Copy code
- name:
  version:
  tags: []
  description:
  schedule: {}
  flow_name:
  entrypoint:
  parameters: {}
  work_pool:
    name:
    work_queue_name:
    job_variables:
      image: '{{ build_image.image }}'
then when you do prefect deploy file:function and pick the docker pool and so on, it creates a second deployment config
Copy code
- name: adamdeployment
  version:
  tags: []
  description:
  entrypoint: flows2.py:pipe2
  parameters: {}
  work_pool:
    name: docker-pool
    work_queue_name:
    job_variables: {}
  schedule:
move the 
job_variables
from the first deployment (the empty unnamed one) to the second deployment, then delete the first deployment
Copy code
deployments:
- name: adamdeployment
  version:
  tags: []
  description:
  entrypoint: flows2.py:pipe2
  parameters: {}
  work_pool:
    name: docker-pool
    work_queue_name:
    job_variables:
      image: '{{ build_image.image }}'
  schedule:
then run 
prefect deploy -n {deploymentname}
to re-register it on cloud
then start the docker-worker and then the deploy command should work fine 🙂
cc @Jeff Hale
a
Thanks @Adam Stone and @Jeff Hale 🙌
n
Thanks I had the same issue!
s
Thanks, but that doesn't work for my use case where I haven't pushed the image to dockerhub, it's just built locally.
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by