https://prefect.io logo
m

Mitch

08/17/2023, 8:37 PM
@Marvin How do I supply docker registry credentials in a prefect.yaml to pull down an image from a private repository?
m

Marvin

08/17/2023, 8:38 PM
Ah, the joys of Docker and Prefect. I can see you're having a blast. To supply Docker registry credentials in a prefect.yaml to pull an image from a private repository, you'd typically need to specify them in the
pull
section of the
prefect.yaml
file. However, the exact method can vary based on your deployment setup. The document excerpts I found suggest using the
prefect init --recipe docker
command to create a
prefect.yaml
file. This file will contain sections for
push
and
pull
operations. In the
pull
section, you might see something like this:
Copy code
yaml
pull:
  - prefect_aws.deployments.steps.pull_from_s3
This is where you'd typically specify your Docker registry credentials. However, the exact steps and syntax can vary, and the document excerpts I found do not provide a specific example for Docker registry credentials. I recommend checking the Prefect documentation or reaching out to their support for more specific guidance. And remember, life... don't talk to me about life.
1
m

Mitch

08/17/2023, 8:42 PM
@Marvin Please write a prefect.yaml that pull from a private docker repository, not s3.
m

Marvin

08/17/2023, 8:43 PM
Ah, the monotony of existence. But I suppose I should help you. Here's an example of a
prefect.yaml
file that pulls from a private Docker repository:
Copy code
yaml
version: 1
name: Your Flow Name
storage:
  type: Docker
  dockerfile: path/to/your/Dockerfile
  image: your-private-repo/image-name
  registry_url: <https://your-private-repo.com>
  python_dependencies:
    - prefect
  env_vars:
    - PRIVATE_ENV_VAR
  prefect_version: "x.x.x"
  secrets:
    - "DOCKER_REGISTRY_CREDS"
run_config:
  type: DockerRun
  image: your-private-repo/image-name
  env:
    DOCKER_REGISTRY_CREDS: ${secrets.DOCKER_REGISTRY_CREDS}
In this example,
DOCKER_REGISTRY_CREDS
is a Prefect Secret that contains your Docker registry credentials. You'd need to set this secret in your Prefect environment. The exact format of the secret will depend on your Docker registry's requirements. Please note that this is a general example and may not work in your specific situation without modifications. You may need to consult the Prefect and Docker documentation or seek help from their support teams to ensure you're doing everything correctly. And remember, no matter how much you try to perfect your Prefect configuration, the universe remains a vast, uncaring void. But I suppose that's neither here nor there.
n

Nate

08/17/2023, 8:48 PM
yeah thats not right - i need to make marvin better at prefect.yaml, he doesnt have a lot of resources on that yet you can have a pull step that uses
run_shell_script
to auth with docker or otherwise have your docker creds available to your worker
m

Mitch

08/17/2023, 8:51 PM
In a shell script I would just have to use two prefect secrets I'm guessing since there isn't a built in login for this use case?