Hi All! I am working on transition to prefect 2 with docker deployment and storage (via the new projects config files) and am running into this error from the worker at runtime:

09:00:08.037 | ERROR  | Flow run 'cobalt-ladybug' - Flow could not be retrieved from deployment.

Based on this recipe on github I saw that “with image-based infrastructure blocks…the image can serve as a fourth storage option…In practice, when creating a deployment with an image-based infrastructure block, you can omit the storage block and Prefect will assume your flow code is available in the image at runtime.” I assume that when i run `prefect deploy`in the CLI, my flows will be bundles up and stored in the docker container, but i guess that is not the case. Can someone provide guidance? I attached my deployment.yaml and prefect.yaml files here to help debug

Hey @Jacob Goldberg! It looks like you’ve commented out your pull step, so prefect will look for your flow code at the default entrypoint of your Docker image. Because you’re using the auto-build feature, your flow code will get stored in a directory matching your project name. If you add this pull step, your deployment should work:

pull:
- prefect.projects.steps.set_working_directory:
    directory: /opt/prefect/calval_etl

Let me know if that works for you!

thanks for the quick help @alex! that makes sense. I will give this a shot and see what happens

Environment variables can also be set via the

job_variables

section in your `deployment.yaml`:

work_pool:
  ...
  job_variables:
    env:
      MY_ENV_VAR: value

The

job_variables

section allows you to override any piece of configuration that is set on the work pool that you’re using for your deployment.

fantastic, thank you! one additional question about this. Those environment variables are actually AWS credentials. I am aware of (and a big fan of ) the new Blocks concept/feature, but for various reasons specific to our infrastructure and setup, it does not make sense to use blocks to provide aws creds directly to the flow/task/functions. Can i use an AWS credentials block to provide creds as environmnet variables here? (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION)

Unfortunately, the casing of the environment variables would be incorrect since the attributes of the AWS credentials block are lowercase. You could use secret blocks instead:

work_pool:
  ...
  job_variables:
    env:
      AWS_ACCESS_KYE_ID: "{{ prefect.blocks.secret.aws-access-key-id }}"
      AWS_SECRET_ACCESS_KEY: "{{ prefect.blocks.secret.aws-secret-access-key }}"
      AWS_DEFAULT_REGION: us-east-1

brilliant! I love the prefect community and support. Would have taken me so much head banging to figure this one out on my own. Ill give this a shot @alex

Thanks for trying out the new projects and deployment experience! Since projects are currently in beta and we’re working on streamlining the experience, let us know if you have any feedback or other issues that you run into while working with your project!

will do! I love the feature. I think the biggest issue i have right now is just a lack of documentation and examples to go off of. But that is also solved by great support and understandable as projects is still in beta! Overall, huge props on this path. I think it is an ideal solution for managing prefect deployments and i love the concept.