https://prefect.io logo
Powered by
Title
k

Kelvin

07/28/2022, 6:21 PM
Hi folks, Does anyone have any luck specifying a schedule in the deployment in 2.0? Also, how to specify environments? in 2.0 beta, I used to use CronSchedule, and SubprocessFlowRunner with a specific environment, but struggling to do the same in 2.0 GA The documentation seems to be missing some details - in the deployment.yaml, there are "schedule" and "infrastructure -> env" fields, but no specifications on what input is required to define them https://orion-docs.prefect.io/concepts/deployments/#deploymentyaml
✅ 1
a

Anna Geller

07/28/2022, 6:27 PM
A much easier way is to add a schedule directly from the UI - I personally think this provides a much better UX. Curious - have you tried that?
k

Kelvin

07/28/2022, 6:42 PM
Ok that works for schedules, thanks Anna! Would you have any pointers on how to use environments also? I used to use SubprocessFlowRunner to run some flows in a separate virtual env
b

Billy McMonagle

07/28/2022, 6:59 PM
Hi @Anna Geller if I may piggyback here... I strongly prefer to set all configuration in code, including schedules... is it possible to do this at this time?
k

Khuyen Tran

07/28/2022, 7:35 PM
Yes you can set configurations in the file 
deployment.yaml
. You can see what you can specify in the file here
:thank-you: 2
a

Anna Geller

07/28/2022, 7:39 PM
Kelvin, here is an example for local process and local file system https://github.com/anna-geller/prefect-deployment-patterns/blob/main/a_project_template/deploy_commands_local_fs.bash
@Billy McMonagle I'll bring this point to the product team
❤️ 1
:thank-you: 1
b

Billy McMonagle

07/28/2022, 7:57 PM
Appreciate it! The UI is super helpful too and the flexibility is great.
:thank-you: 1
🙌 1
k

Kelvin

07/29/2022, 1:46 AM
@Khuyen Tran thanks for the link, but I've already attached it in my original post - that link doesn't give the details on what to put in the "schedule" and "env" (environment) fields, to specify schedules and environments respectively @Anna Geller Thanks for the example, but doesn't "--name local" only change the name to "local" for the deployment? I'm looking for something to replicate what we used to do in 2.0b6 beta and before: 
DeploymentSpec(
    flow_location="myflow.py",
    name="dailyrun",
    schedule=CronSchedule( 
        cron="0 0 * * *",
        timezone="America/New_York"),
    flow_runner=SubprocessFlowRunner(condaenv="specialenv01"),
)
Ok I made some progress, at least for the "using conda env" part I did a bit of "Use the Source, Luke!" and dug into the prefect code I also tried this morning, but seem to be having better luck tonight :-) From what I can see, infrastructure currently supports 3 types https://github.com/PrefectHQ/prefect/blob/20c5f74442e982d5933bfec2f91aa5e14a980f2f/src/prefect/deployments.py 
infrastructure: Union[DockerContainer, KubernetesJob, Process] = Field(
    default_factory=Process
)
Process is defined at: https://github.com/PrefectHQ/prefect/blob/20c5f74442e982d5933bfec2f91aa5e14a980f2f/src/prefect/infrastructure/process.py#L29 "env" only specifies environment variables so not helpful in my case However, it seems that Process runs the command "python" in the yaml - so if I substitute: 
command:
  - python
  - -m
  - prefect.engine
with 
command:
  - /my/path/to/miniconda3/envs/specialenv01/bin/python
  - -m
  - prefect.engine
and deploy a job and run it, it seems to work now
k

Khuyen Tran

07/29/2022, 5:08 PM
We will include information about how to specify schedule in the 
deployment.yaml
file in the docs soon. But for now the easiest way is to use the UI
🙌 1
:thank-you: 1
m

Matt from DataHouse

07/29/2022, 6:44 PM
second this - docs for the scheduler would be a great add!
5 Views
#prefect-communityJoin Slack
Powered by