Matt Alhonte
08/22/2023, 11:41 PMMarvin
08/22/2023, 11:41 PMNate
08/22/2023, 11:43 PMMatt Alhonte
08/22/2023, 11:43 PMNate
08/22/2023, 11:46 PMrepo_root/
project_1/
...
or like
repo_root/
prd/
project_1/
stg/
project_1/
or something else?Matt Alhonte
08/22/2023, 11:47 PMrepo_root/
project_1
and then the individual dev Projects live on different Git branchesNate
08/22/2023, 11:50 PMprefect deploy
from, and attach a corresponding tag to that deployment Matt Alhonte
08/22/2023, 11:51 PMNate
08/22/2023, 11:52 PMMatt Alhonte
08/22/2023, 11:52 PMNate
08/22/2023, 11:56 PMtotally a thingas long as the deployments have different names, so maybe
prefect deploy -n original-name-{branch}
or somethingMatt Alhonte
08/23/2023, 12:18 AM_2
at the end). Tried running the first one, but it fails with this
PermissionError: [Errno 13] Permission denied: './../Hello World-ec2-ec2__cpu_2048__memory_14360_2'
if __name__ == "__main__":
memory = 14360
base_args = make_deployment_args(flow_name, memory, user="matt", tags=["testing2"])
storage = S3Bucket.load(<name>)
deployment1 = Deployment.build_from_flow(
flow=hello_flow, storage=storage, **base_args
)
and then running python flow.py
Nate
08/23/2023, 12:26 AMname
being set in base_args
uniquely for each deployment?Matt Alhonte
08/23/2023, 12:26 AMNate
08/23/2023, 12:26 AMMatt Alhonte
08/23/2023, 12:26 AMec2__cpu_2048__memory_14360
and the other should be ec2__cpu_2048__memory_14360_2
Downloading flow code from storage at 'Hello World-ec2-ec2__cpu_2048__memory_14360'
PermissionError: [Errno 13] Permission denied: './../Hello World-ec2-ec2__cpu_2048__memory_14360_2'
Nate
08/23/2023, 12:30 AMflowname/deployment
for each and what is the storage key for each?Matt Alhonte
08/23/2023, 12:30 AMdef make_deployment_args(
flow_name: str,
memory: int,
user: t.Optional[str] = "dev",
image: t.Optional[
str
],
cpu: t.Optional[int] = None,
tags: t.List[str] = [],
) -> dict:
full_ecs_args = make_ec2_args(memory, user, image, cpu)
ecs_task_block = ECSTask(**full_ecs_args)
resource_string = full_ecs_args["name"]
return {
"infrastructure": ecs_task_block,
"name": resource_string,
"work_queue_name": "default",
"work_pool_name": "default-agent-pool",
"path": f"{flow_name}-ec2-{resource_string}",
"output": f"{flow_name}-ec2-{resource_string}.yaml",
"apply": True,
"tags": tags,
}
version 2, same thing except resource_string = full_ecs_args["name"]
-> resource_string = full_ecs_args["name"] + "_2"
Hello World/ec2__cpu_2048__memory_14360
and Hello World/ec2__cpu_2048__memory_14360_2
Path
is Hello World-ec2-ec2__cpu_2048__memory_14360
, if that's what ya meanNate
08/23/2023, 12:33 AMentrypoint
in the UIMatt Alhonte
08/23/2023, 12:34 AMtags.py:hello_flow
for both of themPath
is Hello World-ec2-ec2__cpu_2048__memory_14360_2
btwNate
08/23/2023, 12:46 AMpath
is the most confusing thing about the infra block paradigm 🙂
personally i like to leave path
alone whenever possible and just set entrypoint
relative to the root of my storage block (but ik sometimes you have to mess with path
)Matt Alhonte
08/23/2023, 12:49 AMNate
08/23/2023, 1:03 AMMatt Alhonte
08/23/2023, 1:05 AMHello World-ec2-ec2__cpu_2048__memory_14360
- any chance that the space is causing trouble?Nate
08/23/2023, 1:07 AMpath
yourself?Matt Alhonte
08/23/2023, 1:07 AMpython flow.py
now. stuff along these lines
00:54:35.216 | DEBUG | hpack.hpack - Decoded (b'permissions-policy', b'accelerometer=(), ambient-light-sensor=(), autoplay=(), battery=(), camera=(), cross-origin-isolated=(), display-capture=(), document-domain=(), encrypted-media=(), execution-while-not-rendered=(), execution-while-out-of-viewport=(), fullscreen=(), geolocation=(), gyroscope=(), hid=(), idle-detection=(), magnetometer=(), microphone=(), midi=(), navigation-override=(), payment=(), picture-in-picture=(), publickey-credentials-get=(), screen-wake-lock=(), serial=(), sync-xhr=(), usb=(), web-share=(), xr-spatial-tracking=()'), total consumed 427 bytes, indexed True
python flow.py >> logs.txt
)Nate
08/23/2023, 1:15 AMMatt Alhonte
08/23/2023, 1:17 AM