Stanislav Kotsiievskyi
10/25/2022, 12:45 PMFlow could not be retrieved from deployment.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 247, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/prefect/deployments.py", line 159, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 146, in get_directory
shutil.copytree(from_path, local_path, dirs_exist_ok=True)
File "/usr/local/lib/python3.10/shutil.py", line 556, in copytree
with os.scandir(src) as itr:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp2ywchethprefect/flows/hello_world'
Have you got experience with such issues?Christopher Boyd
10/25/2022, 1:01 PMAnna Geller
10/25/2022, 1:03 PMStanislav Kotsiievskyi
10/25/2022, 1:48 PMimport sys
import os
from prefect.deployments import Deployment
from prefect.filesystems import GitHub
from prefect.orion.schemas.schedules import CronSchedule
from flow import hello_world_flow
if __name__ == "__main__":
access_token = sys.argv[1]
github_block = GitHub(
repository="<https://github.com/><my-organization>/<my-repo>.git",
reference="my-branch",
access_token=access_token,
)
deployment = Deployment.build_from_flow(
flow=hello_world_flow,
name="hello-world-deployment",
storage=github_block,
work_queue_name="MyQueue",
tags=["test"],
description="flow to test deployment infrastructure",
schedule=CronSchedule(cron="10 13 * * *", timezone="GMT"),
)
# deployment.path = os.path.join("flows", os.path.basename(os.path.dirname(__file__)))
# print(f"deployment.path = {deployment.path}")
deployment.apply()
@Anna Geller
My setup:
1. Agent: Docker container in AWS ECS with prefect agent start -q $PREFECT_WORK_QUEUE
comand in entrypoint
2. Cloud: app.prefect.cloud
3. Deployment: GHA that runs the python script at the top this message.
4. default task_runner (wasn't provided to flow). I believe it would be ConcurrentTaskRunner.Christopher Boyd
10/25/2022, 2:07 PMKhuyen Tran
10/25/2022, 3:53 PMChristopher Boyd
10/25/2022, 5:35 PMawait storage_block.get_directory(from_path=deployment.path, local_path=".")
Stanislav Kotsiievskyi
10/26/2022, 6:40 AMChristopher Boyd
10/26/2022, 12:47 PMDavid Koo
11/11/2022, 8:32 PMFlow could not be retrieved from deployment.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 247, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.8/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 159, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/usr/local/lib/python3.8/site-packages/prefect/filesystems.py", line 144, in get_directory
shutil.copytree(from_path, local_path, dirs_exist_ok=True)
File "/usr/local/lib/python3.8/shutil.py", line 552, in copytree
with os.scandir(src) as itr:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpy6xae0frprefect/flows/My_Script'
Seems like its trying to pull the script from local env instead of from Github. I've tried adding the path but that didn't help with this error. Any ideas would be appreciated. Thanks!Anna Geller
11/12/2022, 5:15 PMDavid Koo
11/14/2022, 4:41 PMsecret_block = Secret.load("my_token")
storage = GitHub(
access_token=secret_block.get(),
reference="branch",
repository="<https://github.com/my_repo.git>",
)
Anna Geller
11/14/2022, 5:35 PM