Sean Malone
12/05/2023, 5:01 PMprefect.yaml
deployments and have 2 types of pull actions I am assigning to YAML aliases (dev/prod). This is the error I get at runtime:
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/deployments/steps/core.py", line 154, in run_steps
step_output = await run_step(step, upstream_outputs)
File "/usr/local/lib/python3.10/site-packages/prefect/deployments/steps/core.py", line 110, in run_step
raise ValueError(
ValueError: Step has unexpected additional keys: repository, access_token, prefect.deployments.steps.git_clone
Here is my prefect.yaml
, am I misusing aliases here?
name: my-prefect
prefect-version: 2.14.9
actions:
pull:
dev: &pull_dev
- prefect.deployments.steps.git_clone:
repository: <https://github.com/my-repo.git>
branch: dev
access_token: "{{ prefect.blocks.secret.my-secret }}"
prod: &pull_prod
- prefect.deployments.steps.git_clone:
repository: <https://github.com/my-repo.git>
branch: main
access_token: "{{ prefect.blocks.secret.my-secret }}"
build: null
push: null
pull: null
# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: my-deploy
version: 1
tags: []
description:
schedule:
entrypoint:
parameters: {}
work_pool:
name:
work_queue_name:
job_variables: {}
pull: *pull_dev # I want this deployment to pull from the dev branch
I am able to successfully run the deployment with a single pull:
definition, but it breaks when trying to use an alias to define the pull action on the deployment pull: *pull_dev
Nate
12/05/2023, 5:26 PMSean Malone
12/05/2023, 5:36 PMactions:
dev: &pull_dev
- prefect.deployments.steps.git_clone:
repository: <https://github.com/my-repo.git>
branch: dev
access_token: "{{ prefect.blocks.secret.my-secret }}"
prod: &pull_prod
- prefect.deployments.steps.git_clone:
repository: <https://github.com/my-repo.git>
branch: main
access_token: "{{ prefect.blocks.secret.my-secret }}"
But am am still getting the “unexpected keys” errorNate
12/05/2023, 5:47 PMdefinitions:
actions:
dev_clone: &dev_clone
- prefect.deployments.steps.git_clone:
repository: <https://github.com/zzstoatzz/prefect-monorepo>
branch: dev
11:45:51.530 | INFO | prefect.deployment - Cloned repository '<https://github.com/zzstoatzz/prefect-monorepo>' into 'prefect-monorepo-dev'
11:45:52.068 | INFO | Flow run 'crouching-flamingo' - Created task run 'some_random_task-0' for task 'some_random_task'
11:45:52.069 | INFO | Flow run 'crouching-flamingo' - Executing 'some_random_task-0' immediately...
11:45:52.430 | INFO | Task run 'some_random_task-0' - Hello from dev
11:45:52.568 | INFO | Task run 'some_random_task-0' - Finished in state Completed()
11:45:52.763 | INFO | Flow run 'crouching-flamingo' - Finished in state Completed('All states completed.')
deployments:
- name: random-flow
entrypoint: src/demo_project/test.py:random_flow
work_pool:
name: local
pull: *dev_clone
Nate
12/05/2023, 5:47 PMSean Malone
12/05/2023, 6:00 PMprefect.deployments.steps.git_clone:
block. I had 2 spaces of indentation, and after increasing it to 4 it ran successfully.
Fixed actions block:
actions:
dev: &pull_dev
- prefect.deployments.steps.git_clone:
repository: <https://github.com/my-repo.git>
branch: dev
access_token: "{{ prefect.blocks.secret.my-secret }}"
prod: &pull_prod
- prefect.deployments.steps.git_clone:
repository: <https://github.com/my-repo.git>
branch: main
access_token: "{{ prefect.blocks.secret.my-secret }}"
Thanks @Nate!