Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
pacc-clearcover-june-12-2023
pacc-may-31-2023
ppcc-may-16-2023
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
r
Revanth
11/09/2022, 2:33 PMHi , is there a way to deploy multiple data pipelines(one for daily and other pipeline for weekly etc) , would appreciate the help , thanks in advance
✅ 1
p
Peyton Runyan
11/09/2022, 3:03 PMTons of ways
You create a deployment for each of the pipelines, and then you set the schedule that you want for each of them.
r
Revanth
11/09/2022, 3:36 PMThank you so much for your help , so basically i woul get multiple yaml files for multiple data pipelines and i just need to deploy them , so let us say i create 3 data pipelines and once i deploy them they would give me 3 sh files , so i just need to run the 3 sh files and everything should work as expected or my understanding was wrong? thanks for the quick responces
p
Peyton Runyan
11/09/2022, 3:53 PMWe don’t create any sh files
How familiar with Prefect are you?
On mobile but I’ll try to answer -
You create your flows with the prefect @flow and @task decorators on your python functions in your data pipeline script.
You then create a deployment, which will produce a YAML file. When you create the deployment you can specify a schedule, or you can add a schedule to a deployment later using the UI.
You the apply the deployment with the CLI command ‘prefect deployment apply <path to deployment yaml>’
Then they will run as long as you have an agent running
If you’re not very familiar with prefect, you should try the tutorial for prefect 2 on our docs page. It will get you up to speed quickly
Then I should be able to help you better when you bump into concepts that are unclear
Gtg for an hour or so, but I’ll check back on this
r
Revanth
11/09/2022, 4:00 PMI apologize i was following an medium article and when the deployment build --apply was applied they showed the output in the shell script and i assumed that prefect would create one my bad , i should have read the article better but thanks for all the help , will get back to you if i have any other queries!
p
Peyton Runyan
11/09/2022, 4:56 PMSounds great and good luck!