I'm trying to setup Prefect using the following docker-compose. Everything seems to work except one important thing: being able to run a flow, which I've deployed with a DockerContainer infrastructure block and Github as storage block. Traceback from the agent (I've removed the url in the second line) :

18:41:58.602 | INFO    | prefect.agent - Submitting flow run '5b4470d3-8cd0-4073-8c03-be70eac1deff'
18:41:59.077 | INFO    | prefect.infrastructure.docker-container - Pulling image URL...
18:41:59.722 | INFO    | prefect.infrastructure.docker-container - Creating Docker container 'invaluable-hyena'...
18:41:59.743 | INFO    | prefect.infrastructure.docker-container - Docker container 'invaluable-hyena' has status 'created'
18:42:00.303 | INFO    | prefect.agent - Completed submission of flow run '5b4470d3-8cd0-4073-8c03-be70eac1deff'
18:42:00.307 | INFO    | prefect.infrastructure.docker-container - Docker container 'invaluable-hyena' has status 'running'
18:42:02.695 | INFO    | prefect.infrastructure.docker-container - Docker container 'invaluable-hyena' has status 'exited'
18:42:02.697 | INFO    | prefect.infrastructure.docker-container - Docker container 'invaluable-hyena' has status 'exited'

Traceback from the container:

Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 260, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/usr/local/lib/python3.8/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 167, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/usr/local/lib/python3.8/site-packages/prefect/filesystems.py", line 908, in get_directory
    process = await run_process(cmd, stream_output=(out_stream, err_stream))
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/processutils.py", line 73, in run_process
    async with open_process(
  File "/usr/local/lib/python3.8/contextlib.py", line 171, in __aenter__
    return await self.gen.__anext__()
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/processutils.py", line 35, in open_process
    process = await anyio.open_process(command, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/anyio/_core/_subprocesses.py", line 127, in open_process
    return await get_asynclib().open_process(
  File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 1105, in open_process
    process = await asyncio.create_subprocess_exec(
  File "/usr/local/lib/python3.8/asyncio/subprocess.py", line 236, in create_subprocess_exec
    transport, protocol = await loop.subprocess_exec(
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 1630, in subprocess_exec
    transport = await self._make_subprocess_transport(
  File "/usr/local/lib/python3.8/asyncio/unix_events.py", line 197, in _make_subprocess_transport
    transp = _UnixSubprocessTransport(self, protocol, args, shell,
  File "/usr/local/lib/python3.8/asyncio/base_subprocess.py", line 36, in __init__
    self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
  File "/usr/local/lib/python3.8/asyncio/unix_events.py", line 789, in _start
    self._proc = subprocess.Popen(
  File "/usr/local/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'git'

It looks like you are close! What image are you using for the flow container? The default image in that repository based on an older version of Prefect, but it's still new enough that it should have Git included - we added it to the image in 2.3.1. But if you are using a different image, you'll need to make sure Git is installed and included in the system path so Prefect can run it as a subprocess to clone your repository. Edit: The image you'll need to check is in your

DockerContainer

infrastructure block. It looks like the deployment example included in the repo uses a custom image that it builds atop

prefecthq/prefect:2.4.5-python3.8

. But if you've changed the image in your deployment I recommend checking there first. 🙂

The DockerContainer infrastructure block was added using the UI so I'm not really sure about the image it is using. Is there a way to check that? Edit: in the Dockerfile of the repo I just mentioned, it is using

FROM prefecthq/prefect:2.4.5-python3.8

. However, my Dockerfile is using

FROM python:latest

. Should I change that to

FROM prefecthq/prefect:2.7.1-python-3.11

?

There's an Image field in the UI you can check. If it is blank, it will try to default to trying to pull an image that matches the Prefect and Python versions running on the agent

Ah, that points to the image on GHCR

Right, now I remember you mentioned the GHCR image a couple of days ago. That image will need Git installed so the GitHub storage block can clone the repository.

It works! Thanks so much for all your help 😄

You are very welcome! I'm happy to hear it's working for you 😄

Unfortunately, I'm still running into an error 😐

Finished in state Failed('Flow run encountered an exception. ValueError: Path /root/.prefect/storage/18f49d2d92c043e187b782512881a0c3 does not exist.\n')

Does this have anything to do with the Github storage block?

/root/.prefect/storage

of course doesn't exist in the repo..