https://prefect.io logo
Channels
Powered by
#prefect-community
Title
# prefect-community
c

Chris Gunderson

05/18/2023, 2:14 PM
We are starting to migrate to Prefect 2. I was setting up the directory structure to have blocks, deployment scripts, flows, tasks, and utilities. The flows folder will have a sub-folder to break up the projects. The same docker image will be used for all project, mainly due to the overlap of libraries we use.
When deploying the flow, I received this error: ValueError: 'C:\\Users\\chris.gunderson\\source\\repos\\prefect2-scripts\\src\\main\\prefect\\flows\\allocations\\prefect_fidelity_allocations.py' is not in the subpath of 'C:\\Users\\chris.gunderson\\source\\repos\\prefect2-scripts\\src\\main\\prefect\\deploy' OR one path is relative and the other is absolute.
image.png
I can move the deployment script to the flows/allocations folder and it works, but I'd like the deployment scripts grouped together. Is that possible?
Copy code
from prefect.deployments import Deployment
from prefect.filesystems import S3
from prefect.orion.schemas.schedules import CronSchedule
from prefect_aws.ecs import ECSTask

from src.main.prefect.flows.allocations.prefect_fidelity_allocations import FidelityAllocationsFlow

s3block = S3.load("sra-s3")
ecs_task_block = ECSTask.load("default")

deployment = Deployment \
    .build_from_flow(name="FidelityAllocationsFlow_Deployment",
                     work_queue_name="ecs-worker",
                     flow=FidelityAllocationsFlow,
                     storage=s3block,
                     schedule=(CronSchedule(cron="13 15 * * 1-5", timezone="America/Chicago")),
                     # Parameters when registering flow to production removed for post
                     infrastructure=ecs_task_block
                     # output="fidelity.yaml" Troubleshooting yaml
                     )
deployment.apply()
n

Nate

05/18/2023, 5:52 PM
hi @Chris Gunderson - if you're just establishing your setup here, i might recommend checking out projects / workers / work pools to handle ECS flow run infra / deployments here's a guide (that will be merged soon) and a rundown of the new concepts
c

Chris Gunderson

05/18/2023, 6:01 PM
@Nate We have already created the AWS template for the Prefect agent, and the configuration for the ECS Tasks. The only thing that I'm looking for is how to separate the flows and deployment scripts. Please see the error and screen shot above.
n

Nate

05/18/2023, 6:17 PM
I understand, I recommended the new approach because i saw
We are starting to migrate to Prefect 2
and projects / workers will eclipse infra blocks / agents as a method of creating deployments in the future projects are built specifically to declaratively group many deployment configs and apply them many at a time, which seemed like what you were driving at as for your error, this highlights a limitation of the older deployment paradigm
Copy code
ValueError: 'C:\\Users\\chris.gunderson\\source\\repos\\prefect2-scripts\\src\\main\\prefect\\flows\\allocations\\prefect_fidelity_allocations.py' is not in the subpath of 'C:\\Users\\chris.gunderson\\source\\repos\\prefect2-scripts\\src\\main\\prefect\\deploy'
your flow file 
is not in the subpath of
your deployment file, which it needs to be when creating a deployment like this
Powered by