Andy Dienes
04/27/2023, 1:40 PMutils
I want to be common to all projects, but i want to develop and deploy projA
and projB
separately. I'm using the s3 recipeflapili
04/27/2023, 1:41 PMflapili
04/27/2023, 1:41 PMAndy Dienes
04/27/2023, 1:42 PMflapili
04/27/2023, 1:42 PMflapili
04/27/2023, 1:42 PMAndy Dienes
04/27/2023, 1:43 PMflapili
04/27/2023, 1:43 PMAndy Dienes
04/27/2023, 1:43 PMAndy Dienes
04/27/2023, 1:43 PMflapili
04/27/2023, 1:44 PMAndy Dienes
04/27/2023, 1:44 PMutils
into each project subdir, so code duplication also = fail 🙂flapili
04/27/2023, 1:45 PMflapili
04/27/2023, 1:45 PMflapili
04/27/2023, 1:45 PMflapili
04/27/2023, 1:46 PMAndy Dienes
04/27/2023, 1:46 PMsrc
-flows
-project_1
deployment.yaml
flow1.py
-project_2
deployment.yaml
flow2.py
-utils
file.py
Andy Dienes
04/27/2023, 1:46 PMimport file
?Andy Dienes
04/27/2023, 1:47 PMflapili
04/27/2023, 1:47 PMflapili
04/27/2023, 1:47 PMflapili
04/27/2023, 1:47 PMflapili
04/27/2023, 1:48 PMAndy Dienes
04/27/2023, 1:48 PMflapili
04/27/2023, 1:48 PMAndy Dienes
04/27/2023, 1:48 PMutils
in the push
block?Andy Dienes
04/27/2023, 1:48 PMAndy Dienes
04/27/2023, 1:51 PMsrc/utils
in the same repo? not sure how to make that work with projectsflapili
04/27/2023, 1:56 PMalex
04/27/2023, 2:04 PMAndy Dienes
04/27/2023, 2:07 PMAndy Dienes
04/27/2023, 2:08 PMredsquare
04/27/2023, 2:09 PMredsquare
04/27/2023, 2:10 PMAndy Dienes
04/27/2023, 2:11 PMprefect.yaml
or the deployment.yaml
or are you doing it manually?redsquare
04/27/2023, 2:11 PMredsquare
04/27/2023, 2:12 PMredsquare
04/27/2023, 2:12 PMredsquare
04/27/2023, 2:12 PMredsquare
04/27/2023, 2:13 PMalex
04/27/2023, 2:13 PMgit_clone_project
step. It sounds like you’d need multiple deployments for this project, and each deployment would have a different entrypoint to the flow that it wraps. A UX for declaring multiple deployments in a project should be out later today, but you can check out this PR to see what the experience will be like.Andy Dienes
04/27/2023, 2:17 PMyou should be able to pull it down with a singleso this would go in mystepgit_clone_project
pull
section? I'm using the s3
recipe so right now it's pulling from s3Andy Dienes
04/27/2023, 2:17 PMalex
04/27/2023, 2:19 PMAndy Dienes
04/27/2023, 2:24 PMAndy Dienes
04/27/2023, 2:25 PMAndy Dienes
04/27/2023, 2:27 PMAndy Dienes
04/27/2023, 2:27 PMflow1.py
and all of utils
, and the second deployment would push only flow2.py
and all of utils
alex
04/27/2023, 2:31 PMpush
and pull
steps so you could push and pull each deployment to different locations. Each deployment would also need its own .prefectignore
file to prevent uploading other flow files during deployment.
It’s possible, but it also introduces additional complexity. What problem are you solving for by excluding files on a per-deployment basis?