Hi , is there a way to deploy multiple data pipeli...
# prefect-community
r
Hi , is there a way to deploy multiple data pipelines(one for daily and other pipeline for weekly etc) , would appreciate the help , thanks in advance
1
p
Tons of ways
You create a deployment for each of the pipelines, and then you set the schedule that you want for each of them.
r
Thank you so much for your help , so basically i woul get multiple yaml files for multiple data pipelines and i just need to deploy them , so let us say i create 3 data pipelines and once i deploy them they would give me 3 sh files , so i just need to run the 3 sh files and everything should work as expected or my understanding was wrong? thanks for the quick responces
p
We don’t create any sh files
How familiar with Prefect are you?
On mobile but I’ll try to answer - You create your flows with the prefect @flow and @task decorators on your python functions in your data pipeline script. You then create a deployment, which will produce a YAML file. When you create the deployment you can specify a schedule, or you can add a schedule to a deployment later using the UI. You the apply the deployment with the CLI command ‘prefect deployment apply <path to deployment yaml>’ Then they will run as long as you have an agent running
If you’re not very familiar with prefect, you should try the tutorial for prefect 2 on our docs page. It will get you up to speed quickly
Then I should be able to help you better when you bump into concepts that are unclear
Gtg for an hour or so, but I’ll check back on this
r
I apologize i was following an medium article and when the deployment build --apply was applied they showed the output in the shell script and i assumed that prefect would create one my bad , i should have read the article better but thanks for all the help , will get back to you if i have any other queries!
p
Sounds great and good luck!
119 Views