Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Untitled

Hello. I use pull_from_s3 step and sometimes I see the following error:

Image that I use has `prefect-aws` installed

prefect.yaml:
```build: null
push:
  - prefect_aws.deployments.steps.push_to_s3:
      id: push-flow
      requires: prefect-aws==0.3.6
      bucket: org-prefect-poc
      folder: flows/sample

pull:
  - prefect_aws.deployments.steps.pull_from_s3:
      id: pull-flow
      bucket: org-prefect-poc
      folder: "{{ push-flow.folder }}"

deployments:
  - name: sample
    entrypoint: sample.py:sample
    work_pool:
        name: default-k8s-pool
        work_queue_name: k8s-queue
        job_variables:
          service_account_name: sample
          env:
            EXTRA_PIP_PACKAGES: org-chronos[complete]==1.14.4
    schedule: null```

and there is one more error that is random as well:

interesting fact about the flow runs where I see these errors: there is no mention about flow run in worker logs and there is no pod yet. I see errors immediately on UI.

&gt; ```botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied```
this is telling you that whatever credentials you are using don’t have the appropriate permissions to complete the api call

I don’t see where you specify the image to use. By default, the worker will use a prefect image which doesn’t have the `prefect_aws` module installed on it

<@U02L949MUGM> K8s pod has correct role with needed permissions. I see this error even before pod started. Looks like some other machine is trying to run pull from s3 step. I think these errors are related.

image.png

<@U02L949MUGM> Could you please help me to understand how it works?