Heya! So, we used to have individual "dev" Project...
# ask-community
m
Heya! So, we used to have individual "dev" Projects in Prefect 1, and a separate production Project. So like there's a version of Flow X on my dev Project, and another version of Flow X on the production Project (and ya can eventually merge changes from one into the other). How do we approximate this with the new Tagging system with just one Workspace? Can you have different Deployments of the same Flow that have different code for the flow? Do ya do it purely with tagging Tasks? Or will individual versions of flows need different names?
j
Hi @Matt Alhonte! classic answer for you is it depends πŸ™‚ you can definitely have multiple deployments (one for dev and one for prod) that share the same underlying flow code and pass different parameters to each, have them run in separate work pools (cloud environments), etc. If you want to test the flow code before merging to main, you could setup your deployment to pull from a β€œdev” like branch, or deploy the deployment locally
also of course you could separate by workspace which would allow you the most separation between environments
m
@Jamie Zieziula Can the underlying code be different though? ie, like literally different code in the body of the Flow or its Tasks?
Would rather not spin up different Workspaces due to increased costs & complexity (multiple keys that need to be piped into various other systems, for instance)
j
Do you mean different code like totally different code? or changes you want to test in a dev environment?
m
Including totally different code, yeah
Same Flow name, same name of Flow function, different code.
Makes it a lot easier to merge changes
j
are both versions of those code meant to live on main?
m
No, they'd live on different git branches
j
oh i see
and ya can eventually merge changes from one into the other
yep so you can do as i mentioned above - youll have two deployments, with the associated env-specific configuration. youll need the pull step of your dev deployment to point to the dev-branch & the pull step of your pd deployment to point to the prd-branch
m
The way we did it in Prefect 1 was you'd make a change on your branch -> CI/CD runs & rebuilds your Image & re-registers your versions of the flows onto your Project
j
you could accomplish the same now, it might just take a bit more cicd magic to get it to work for your dev branches (unless you have a long lived branch)
the merge to main workflow should be simple though
m
@Jamie Zieziula Does that work well with deploying the
script
way? Right now the Prefect 2 workflow has something like this at the bottom of every Flow file
Copy code
for memory in [15360, 30720, 63398, 126899]:
        base_args = make_deployment_args(flow_name1, memory, user="matt")
        storage1 = S3Bucket.load(<name>)
        deployment1 = Deployment.build_from_flow(
            flow=nb_flow, storage=storage1, **base_args
        )
j
okay sorry sorry, i should have asked! i assumed you were using the new version of deployments & workers
m
For the test I'm running, I've got one Deployment called
ec2__cpu_2048__memory_14360
and one called
ec2__cpu_2048__memory_14360_2
j
i am not as familiar with that deployment methodology, ill ask a peer of mine if they can way in
πŸ‘ 1
m
When I try to run the first one, it fails with
PermissionError: [Errno 13] Permission denied: './../Hello World-ec2-ec2__cpu_2048__memory_14360_2'