<@ULVA73B9P> Does Prefect 3 not include newly-crea...
# ask-marvin
m
@Marvin Does Prefect 3 not include newly-created deployments in prefect.yaml by default? I've been using code-based deployments for a while, but just tried going back to creating one via "prefect deploy". It created it successfully in my prefect instance, but doesn't prompt me to see if I want to save the deployment code for later, and doesn't edit prefect.yaml
m
thought for 116.6 seconds
Yes — in Prefect 3.x,
prefect deploy
does not always write back to
prefect.yaml
by default. What it does now is: - If no
prefect.yaml
exists and you're in an interactive terminal,
prefect deploy
should prompt: -
Would you like to save configuration for this deployment for faster deployments in the future?
- If you say yes, it creates
prefect.yaml
- If
prefect.yaml
already exists, Prefect uses it, but does not generally prompt again to save/update it interactively in the same way So if you saw: - deployment created successfully in Cloud/server - no prompt to save - no changes to
prefect.yaml
that is consistent with one of these situations: 1. You already had a
prefect.yaml
in the project/root it was using 2. The command was not considered interactive 3. You were deploying in a way that bypassed the first-time save flow A few important details: -
prefect deploy
is still the correct CLI command in 3.x - Code-based deployments via
.deploy()
/
flow.from_source(...).deploy(...)
are a separate workflow and do not rely on
prefect.yaml
-
prefect.yaml
is mainly for YAML/project-based deployment configuration, not something Prefect guarantees it will continuously auto-sync from server-created deployments So the short answer is: No, newly created deployments are not automatically added to
prefect.yaml
by default in all cases.
If you want a reusable YAML-based setup, the intended pattern is usually: - initialize/configure your project files - keep deployments in
prefect.yaml
- use
prefect deploy
/
prefect deploy --all
from that file Useful references: - prefect.yaml docs - Deploy via Python - Deployment guides If you want, I can also help you figure out why your specific run did not prompt — for example by checking whether Prefect was picking up an existing
prefect.yaml
, or by walking through the exact command you ran.
m
@Marvin if I have an existing prefect.yaml (created by "prefect init", with no deployments specified), shouldn't it pick that up and prompt me to ask if I want to save? I'm running from an interactive terminal.
m
thought for 8.3 seconds
Not currently, no. In Prefect 3.x, the “save this deployment for later” prompt is only shown when no
prefect.yaml
exists at all
. If you already have a
prefect.yaml
— even one created by
prefect init
with an empty
deployments:
section or no deployments configured yet —
prefect deploy
treats that as an existing project config and skips the first-time save prompt. So your case: -
prefect.yaml
exists - no deployments are defined yet - interactive terminal => Prefect will not prompt you to save the new deployment into that file That behavior appears to be based on a simple “does the file exist?” check, not “does this file already contain deployment definitions?”. So if you want the deployment persisted in
prefect.yaml
, you currently need to add it there yourself. A couple of related notes: -
prefect deploy
can still create the deployment in Prefect Cloud/server from CLI inputs - but it won’t automatically round-trip that generated config back into an already-existing
prefect.yaml
- if you want YAML-managed deployments, it’s best to treat
prefect.yaml
as the source of truth and define/edit deployments there before re-running
prefect deploy
If you want, I can give you a minimal
prefect.yaml
deployment stanza you can paste in for your flow.
^ would something like this be useful to you?
m
@Marvin when did this change? it used to prompt me any time there was a new deployment added, and then run without prompting on future runs. Is there another CLI utility that we can use to quickly add things with the proper schema? @Nate I think so. The last time I used the CLI for creating deployments was in the 2.x days, where I remember it adding them for me.
m
thought for 213.4 seconds
Yes — your memory is basically right, but there’s one important correction: - In Prefect 2.x, the workflow was different (
prefect deployment build
/
prefect deployment apply
) - The behavior you’re remembering with
prefect deploy
prompting to save new deployments into
prefect.yaml
was actually early Prefect 3.x behavior, not 2.x When did it change? It looks like this changed in Prefect 3.3.4 via PR
#17519
with the change described as: - “Only prompt to save if file does not exist” So before that change, interactive
prefect deploy
would prompt more broadly when creating a deployment and could add/update entries in
prefect.yaml
. After that change, the prompt only appears if: - you are interactive, and -
prefect.yaml
does not already exist So your current experience is expected in newer 3.x releases. Is there another CLI utility to quickly add deployment entries with the proper schema? Short answer: not really. There is not currently a dedicated CLI like “add this deployment to `prefect.yaml`” for an existing file. Your current options are: 1. Manual edit of
prefect.yaml
2. Use
prefect init
to scaffold a project/recipe, but not to append deployment entries interactively 3. Use Python-based deployments (
flow.from_source(...).deploy(...)
or
flow.deploy(...)
) instead of YAML-managed deployments
prefect init
still exists, but it’s for project scaffolding, not for interactively inserting new deployment blocks into an existing
prefect.yaml
. So if you want the “quickly add a correctly-shaped deployment entry” experience, today the closest thing is either: - create a fresh project scaffold and copy the generated structure, or - manually add a deployment block to
prefect.yaml
A minimal example looks like:
Copy code
yaml
deployments:
  - name: my-deployment
    entrypoint: flows/my_flow.py:my_flow
    work_pool:
      name: my-work-pool
Then run:
Copy code
bash
prefect deploy --name my-deployment
or just:
Copy code
bash
prefect deploy
depending on how many deployments are defined. Relevant docs: - prefect.yaml deployments - Create deployments - Deploy via Python If you want, I can generate a copy-pasteable deployment block for your flow with the current 3.x schema — just send me: - your flow entrypoint - deployment name - work pool name - optional schedule / parameters / tags
n
2.x shouldn't have been that different, unless you had some custom harness for the CLI (unless there's a branch of the interactive CLI that i'm forgetting, which is possible) editing a shared/checked-in
prefect.yaml
and then
prefect --no-prompt deploy --all
in CI is my general recommendation the thing that often trips people up is that
prefect.yaml
isn't the exhaustive state on the server, its just a declaration of what you want to exist when you run
prefect deploy
, but someone else might have their own
specific-prefect.yaml
file. some orgs like different norms, ie certain teams have their own
prefect.yaml
vs strictly keeping all things in one
prefect.yaml
per server/workspace
m
@Nate That should work well for CI/CD. I'm just trying to find a way to guide some new team members through that "cli wizard" style process for picking the entrypoint, generating schedule, etc. We' re looking to consolidate a bunch of single-workflow repos into one big monorepo, and it looked like going back to the YAML setup and CLI may be a better option than our custom code deployment. It'd be nice to be able to trigger that "wizard" and save the output as well-formatted YAML whenever they add a new pipeline to the repo
n
gotcha! yea
It'd be nice to be able to trigger that "wizard" and save the output as well-formatted YAML whenever they add a new pipeline to the repo
this should happen today? ie if your
prefect.yaml
exists, and you specify a new deployment by answering the wizard's questions, it should ask you to persist it as a new entry in the yaml.. if it doesn't that'd be unexpected to me and i can look into it!
m
@Nate ahh ok that's where I'm confused. because it's generating a deployment in my local test server (local prefect OSS instance running 3.7.1), but not prompting me to save the output into Prefect.yaml, and not modifying the deployments section. That's where I was confused, because it definitely always saved it in there before. it creates the deployment, but doesn't touch prefect.yaml: Successfully built 23a6a1b41984
prefect._experimental._launchers
has moved to
prefect._internal.launchers
. The old import path will be removed in a future release. ? Would you like to configure schedules for this deployment? [y/n] (y): y ? What type of schedule would you like to use? [Use arrows to move; enter to select] ┏━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ┃ Schedule Type ┃ Description ┃ ┡━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ │ Interval │ Allows you to set flow runs to be executed at fixed time intervals. │ │ > │ Cron │ Allows you to define recurring flow runs based on a specified pattern using cron syntax. │ │ │ RRule │ Allows you to define recurring flow runs using RFC 2445 recurrence rules. │ └────┴───────────────┴──────────────────────────────────────────────────────────────────────────────────────────┘ ? Cron string (0 0 * * *): 0 8 * * * ? Timezone (UTC): America/Regina ? Would you like to activate this schedule? [y/n] (y): y ? Would you like to add another schedule? [y/n] (n): n ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ Deployment 'accounting_files_2/default' successfully created with id '5bbccb5e-d2e4-42fb-8aa7-99ae4526db6d'.
Kicked it off just doing a "prefect deploy", picked the entrypoint, created 1 schedule, answred "Y" to activate the schedule, "N" to create another schedule. Then exits out with a success message without saving. Prefect.yaml stays like this:
Copy code
# the deployments section allows you to provide configuration
# for deploying flows
deployments:
- name: null
  version: null
  tags: []
  description: null
  schedule: {}
  flow_name: null
  entrypoint: null
  parameters: {}
  work_pool:
    name: null
    work_queue_name: null
    job_variables:
      image: '{{ build_image.image }}'
There is one warning near the end: "`prefect._experimental._launchers` has moved to
prefect._internal.launchers
. The old import path will be removed in a future release.". This is a new repo, just added Prefect>=3.7.1, no customizations that I'm aware of.
n
thanks! helpful context, will take a look
1
m
@Nate don't know if I'm reading this correctly, but I think a regression may have snuck in when there was a large-scale migration done on the CLI. It seems to only prompt if you want to save the configuration now if the prefect_file (default prefect.yaml) doesn't exist: (prefect/cli/deploy/_core.py, line 423): if is_interactive() and not prefect_file.exists(): if confirm( ( "Would you like to save configuration for this deployment for faster" " deployments in the future?" ), console=console, ): deploy_config_before_templating.update({"schedules": _schedules}) _save_deployment_to_prefect_file( deploy_config_before_templating, build_steps=build_steps or None, push_steps=push_steps or None, pull_steps=pull_steps or None, triggers=trigger_specs or None, sla=sla_specs or None, prefect_file=prefect_file, ) console.print( ( f"\n[green]Deployment configuration saved to {prefect_file}![/]" " You can now deploy using this deployment configuration" " with:\n\n\t[blue]$ prefect deploy -n" f" {deploy_config['name']}[/]\n\nYou can also make changes to" " this deployment configuration by making changes to the" " YAML file." ), ) The method it calls (_save_deployment_to_prefect_file) already seems to check if the file exists, and gracefully handles creating or editing it, so I'm not sure if that exists() call is necessary? "def _save_deployment_to_prefect_file( deployment: dict[str, Any], build_steps: list[dict[str, Any]] | None = None, push_steps: list[dict[str, Any]] | None = None, pull_steps: list[dict[str, Any]] | None = None, triggers: list[dict[str, Any]] | None = None, sla: list[dict[str, Any]] | None = None, prefect_file: Path = Path("prefect.yaml"), ): """ Save a deployment configuration to the
prefect.yaml
file in the current directory. Will create a prefect.yaml file if one does not already exist. Args: - deployment: a dictionary containing a deployment configuration """ deployment = _format_deployment_for_saving_to_prefect_file(deployment) current_directory_name = os.path.basename(os.getcwd()) if not prefect_file.exists(): if triggers: deployment["triggers"] = triggers if sla: deployment["sla"] = sla create_default_prefect_yaml( ".", current_directory_name, contents={ "deployments": [deployment], "build": build_steps, "push": push_steps, "pull": pull_steps, }, ) create_default_ignore_file(".") else: # use ruamel.yaml to preserve comments ryaml = YAML() ... proceeds to read the existing config, merge with new config, then write the new file "
n
thanks for the thoughts! i think this an edge case more likely when using
prefect init
that arose from an overly broad heuristic to stop extra prompts. will get this reviewed and update here!
👏 1