https://prefect.io logo
Channels
Powered by
c

Cody

07/26/2023, 10:27 PM
@Marvin Why am I hitting this error prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect_docker.deployments.steps.push_docker_image
Traceback (most recent call last): File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\deployments\steps\core.py", line 124, in run_steps step_output = await run_step(step, upstream_outputs) File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\deployments\steps\core.py", line 95, in run_step result = await from_async.call_soon_in_new_thread( File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\_internal\concurrency\calls.py", line 292, in aresult return await asyncio.wrap_future(self.future) File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\_internal\concurrency\calls.py", line 316, in _run_sync result = self.fn(*self.args, **self.kwargs) File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect_docker\deployments\steps.py", line 300, in push_docker_image raise OSError(event["error"]) OSError: denied: requested access to the resource is denied The above exception was the direct cause of the following exception: Traceback (most recent call last): File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\cli\_utilities.py", line 41, in wrapper return fn(*args, **kwargs) File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\utilities\asyncutils.py", line 255, in coroutine_wrapper return call() File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\_internal\concurrency\calls.py", line 383, in call return self.result() File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\_internal\concurrency\calls.py", line 283, in result return self.future.result(timeout=timeout) File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\_internal\concurrency\calls.py", line 169, in result return self.__get_result() File "C:\Users\Cody\anaconda3\envs\prefect2\lib\concurrent\futures\_base.py", line 403, in __get_result raise self._exception File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\_internal\concurrency\calls.py", line 346, in _run_async result = await coro File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\cli\deploy.py", line 255, in deploy await _run_single_deploy( File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\client\utilities.py", line 51, in with_injected_client return await fn(*args, **kwargs) File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\cli\deploy.py", line 515, in _run_single_deploy await run_steps(push_steps, step_outputs, print_function=app.console.print) File "C:\Users\Cody\anaconda3\envs\prefect2\lib\site-packages\prefect\deployments\steps\core.py", line 152, in run_steps raise StepExecutionError(f"Encountered error while running {fqn}") from exc prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect_docker.deployments.steps.push_docker_image
n

Nate

07/26/2023, 10:28 PM
have you authed to your docker registry?
c

Cody

07/26/2023, 10:29 PM
My image is in AWS ECR, which I am authenticated with. Not sure if prefect currently knows that fact though
n

Nate

07/26/2023, 10:31 PM
hmm seems like you're running locally, so if you're authed locally docker should find / use those creds
c

Cody

07/26/2023, 10:36 PM
I reran the usual 
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin blah-blah
to make sure that I'm authed with ECR. I can push images there just fine. Higher up I see
Copy code
Running deployment push steps...
 > Running push_docker_image step...
The push refers to repository [<http://docker.io/library/trigger-flow|docker.io/library/trigger-flow>]
I'm not using docker.io for anything here to my knowledge. docker.io ~= AWS ecr, no? I'm guessing I need to set prefect.yaml to point at my ecr, but thats just a guess
n

Nate

07/26/2023, 10:37 PM
oh
c

Cody

07/26/2023, 10:37 PM
I may also be using the wrong pattern. I want my deployment to basically grab an ECR image and create an ECS task to run the Prefect task on, where the code lives in the ECR image
n

Nate

07/26/2023, 10:37 PM
your image needs a repo
in your prefect.yaml
Copy code
reponame/image
so like
Copy code
<numbers>.dkr.ecr.<region>.<http://amazonaws.com/trigger-flow|amazonaws.com/trigger-flow>
c

Cody

07/26/2023, 10:41 PM
That's under
Copy code
build:
- prefect_docker.deployments.steps.build_docker_image:
  image_name: <here>
right?
Also, I appreciate your time
n

Nate

07/26/2023, 10:41 PM
yep! except its just 
image
iirc lemme check
c

Cody

07/26/2023, 10:42 PM
This is the entirety of that section from the prefect init
Copy code
# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
    requires: prefect-docker>=0.3.1
    image_name: trigger-flow
    tag: latest
    dockerfile: Dockerfile
Currently running prefect 2.11 if that matters
I got it, it looks
n

Nate

07/26/2023, 10:46 PM
oh sorry my last response was confusing, you can add the repo to your 
image
name
Copy code
# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
    requires: prefect-docker>=0.3.1
    image_name: <numbers>.dkr.ecr.<region>.<http://amazonaws.com/trigger-flow|amazonaws.com/trigger-flow>
    tag: latest
    dockerfile: Dockerfile
push:
- prefect_docker.deployments.steps.push_docker_image:
    requires: prefect-docker>=0.3.1
    image_name: "{{ build-image.image_name }}"
    tag: "{{ build-image.tag }}"
and add a push step
oh ok cool
c

Cody

07/26/2023, 10:48 PM
still pushing, but looks hopeful. Putting
Copy code
# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
    requires: prefect-docker>=0.3.1
    image_name: <numbers>.dkr.ecr.<region>.<http://amazonaws.com/trigger-flow:latest|amazonaws.com/trigger-flow:latest>
    tag: latest
    dockerfile: Dockerfile
Was what got me there
Ahhh, I was thinking you wanted me to add a new 
image
field. The image gets pulled just fine with the updates I wanted.
n

Nate

07/26/2023, 10:50 PM
just a nit, you prob dont wanna duplicate the tag like that
Copy code
image_name: <numbers>.dkr.ecr.<region>.<http://amazonaws.com/trigger-flow:latest|amazonaws.com/trigger-flow:latest>
instead prob just
Copy code
image_name: <numbers>.dkr.ecr.<region>.<http://amazonaws.com/trigger-flow|amazonaws.com/trigger-flow>
yeah haha sorry that was a confusing thing to say, i was thinking of 
image
bc thats the name you grab downstream in later steps
c

Cody

07/26/2023, 10:52 PM
right, makes sense. I'll make that tag change, that also makes sense lol. Does the 
push
section you suggested push the image to ECR automatically? If so thats kinda awesome
n

Nate

07/26/2023, 10:55 PM
yes!
Powered by