Hi, I'm running prefect 2.6.3. I have an agent in docker container running as ECS service and Github filesystem with private repo to store my flows. The flows are in 'flows' subfolder in repo. I'm getting this traceback when trying to run the flow:

Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 247, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/deployments.py", line 159, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 146, in get_directory
    shutil.copytree(from_path, local_path, dirs_exist_ok=True)
  File "/usr/local/lib/python3.10/shutil.py", line 556, in copytree
    with os.scandir(src) as itr:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp2ywchethprefect/flows/hello_world'

Have you got experience with such issues?

HI Stanislav, I’ve seen this reported a few times , if you can help me get some details, i can file an issue for it

+1 I'm curious about your setup - is this ECSTask infra block + GitHub storage block + agent running inside a Docker container in a local process?

@Christopher Boyd Here is how I'm creating the deployment. I'm using here an anonymous infrastructure here, but also tried with block created in UI.

import sys

import os

from prefect.deployments import Deployment

from prefect.filesystems import GitHub

from prefect.orion.schemas.schedules import CronSchedule

from flow import hello_world_flow

if __name__ == "__main__":

access_token = sys.argv[1]

github_block = GitHub(

repository="<https://github.com/><my-organization>/<my-repo>.git",

reference="my-branch",

access_token=access_token,

)

deployment = Deployment.build_from_flow(

flow=hello_world_flow,

name="hello-world-deployment",

storage=github_block,

work_queue_name="MyQueue",

tags=["test"],

description="flow to test deployment infrastructure",

schedule=CronSchedule(cron="10 13 * * *", timezone="GMT"),

)

# deployment.path = os.path.join("flows", os.path.basename(os.path.dirname(__file__)))

# print(f"deployment.path = {deployment.path}")

deployment.apply()

@Anna Geller My setup: 1. Agent: Docker container in AWS ECS with

prefect agent start -q $PREFECT_WORK_QUEUE

comand in entrypoint 2. Cloud: app.prefect.cloud 3. Deployment: GHA that runs the python script at the top this message. 4. default task_runner (wasn't provided to flow). I believe it would be ConcurrentTaskRunner.

gotcha, thank you

@Stanislav Kotsiievskyi To make sure this is nothing to do with the infrastructure you are running, can you try to run the agent in your local machine?

Okay, I think for your issue, is where are you executing the code?

@Christopher Boyd adding path to the deployment worked, thanks! Also there was an issue with authorization, that i missed because the error was "FileNotFoundError"

awesome!

Hello, I work with @Stanislav Kotsiievskyi and been experiencing the same issue here. What I'm trying to do is create the Github storage programmatically & be able to take a branch name as a parameter. My code looks very similar to https://prefect-community.slack.com/archives/CL09KU1K7/p1666705712583439?thread_ts=1666701947.325439&cid=CL09KU1K7 and when deployed thru GHA, I get the follow error:

Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 247, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/usr/local/lib/python3.8/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 159, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/usr/local/lib/python3.8/site-packages/prefect/filesystems.py", line 144, in get_directory
    shutil.copytree(from_path, local_path, dirs_exist_ok=True)
  File "/usr/local/lib/python3.8/shutil.py", line 552, in copytree
    with os.scandir(src) as itr:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpy6xae0frprefect/flows/My_Script'

Seems like its trying to pull the script from local env instead of from Github. I've tried adding the path but that didn't help with this error. Any ideas would be appreciated. Thanks!

David, this post might help https://medium.com/the-prefect-blog/modular-data-stack-build-a-data-platform-with-prefect-dbt-and-snowflake-part-6-7e6d69296649

Thank you @Anna Geller for the post, super helpful! It explicitly states that when creating a Github block in python code for private repo, it needs a PAT stored in .env file coming from root directory. This is a problem for us as we don't want to store Access Token in the repo. I've already tried creating a secret block thru UI and pulling it into the python code but it wasn't working for me. To your knowledge, is this possible?

secret_block = Secret.load("my_token")
    storage = GitHub(
        access_token=secret_block.get(),
        reference="branch",
        repository="<https://github.com/my_repo.git>",
    )

did you try to follow the blog post?