Tibs
01/05/2023, 5:29 AMredsquare
01/05/2023, 7:15 AMBen Muller
01/05/2023, 8:01 AMprefect build apply
commands concurrently with the &
command but for a repo with 20ish deployments it can take about 10 minutes per environment ( prod | staging ) to deploy. It has been a massive time suck.
Would be great to get a batch deploy or something of the kind...Tibs
01/05/2023, 8:07 AMredsquare
01/05/2023, 8:12 AMTibs
01/05/2023, 8:20 AMredsquare
01/05/2023, 8:20 AMBen Muller
01/05/2023, 8:35 AMredsquare
01/05/2023, 8:37 AMAnna Geller
Tibs
01/05/2023, 1:36 PMAnna Geller
Tibs
01/05/2023, 1:40 PMAnna Geller
Tibs
01/05/2023, 1:48 PMBen Muller
01/05/2023, 3:45 PMAnna Geller
Ben Muller
01/05/2023, 6:19 PMAnna Geller
--path /opt/prefect
Ben Muller
01/05/2023, 7:35 PMAnna Geller
Ben Muller
01/05/2023, 8:16 PMsave
command ?
I can not see a build command on the ECSTask
block?
Or do you mean the path on the prefect deployment build
command ?Anna Geller
prefect deployment build myflow.py:myflow -ib ecs-task/default --path /opt/flows -n default -a
Ben Muller
01/05/2023, 9:20 PM--storage-block
argument or it will look in s3 by default, yeah?Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 262, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/prefect/deployments.py", line 166, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 143, in get_directory
copytree(from_path, local_path, dirs_exist_ok=True)
File "/usr/local/lib/python3.10/shutil.py", line 556, in copytree
with os.scandir(src) as itr:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/flows'
For the record it only saved 525 seconds ~ in deploy time per environment ( I am deploying 12 flows )
The costly part seems to be the generation of the yml file in the build
command--path
options ?
--path {work_dir}/flows/my_flow
?Tibs
01/10/2023, 10:18 AM