<@ULVA73B9P> Im getting an error after deploying t...
# ask-marvin
f
@Marvin Im getting an error after deploying this code
Copy code
import os
import json
import sys
from prefect.flows import load_flow_from_entrypoint
from prefect.docker.docker_image import DockerImage
from prefect.filesystems import LocalFileSystem


def main():
    if len(sys.argv) != 2:
        print("Usage: python deploy.py <config_path>")
        sys.exit(1)
    
    config_path = sys.argv[1]
    
    # Load configuration
    with open(config_path, "r") as f:
        config = json.load(f)

    # add commit tag
    tags = config.get("tags", [])
    commit = os.getenv("GITHUB_SHA")
    if commit:
        tags.append(f"commit:{commit[:8]}")

    # load flow from entrypoint
    flow = load_flow_from_entrypoint(config["entrypoint"])


    docker_image = DockerImage(
        name="cyclopsai/template",  
        dockerfile=config["docker_image"],
    )
    
    # deploy directly from source
    flow.deploy(
        name=config["name"],
        work_pool_name=config["work_pool_name"],
        description=config.get("description", ""),
        tags=tags,
        push=True,
        image=docker_image,
        parameters=config.get("parameters", {}),
    )


if __name__ == "__main__":
    main()
the docker image is pushed to registry correctly. however when I run the deployment I get the following error:
Copy code
Downloading flow code from storage at None
02:53:35 PM
prefect.flow_runs
Runner failed to retrieve flow to execute on_crashed hooks for flow run UUID('0fd0810a-d5bf-45e7-8703-30e363338b24').
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/prefect/runner/runner.py", line 1548, in _run_on_crashed_hooks
    flow = await load_flow_from_flow_run(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/client/utilities.py", line 69, in wrapper
    return await func(client, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prefect/flows.py", line 2806, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=from_path, local_path=".")
  File "/usr/local/lib/python3.12/site-packages/prefect/filesystems.py", line 162, in aget_directory
    copytree(from_path, local_path, dirs_exist_ok=True, ignore=ignore_func)
  File "/usr/local/lib/python3.12/shutil.py", line 598, in copytree
    with os.scandir(src) as itr:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/workspace/None'
m
thought for 521.6 seconds
Thanks for the details — the error makes sense given the current deploy script. What’s happening - Your deployment is being created without any source/storage info, so at runtime the worker doesn’t know where to find your flow code. - That’s why you see “Downloading flow code from storage at None” and then a FileNotFoundError like “/workspace/None”. - In Prefect 3.x, a deployment must include instructions for how the worker will get the code at run time (aka “pull steps”). You provide these by calling
Flow.from_source(...)
before
.deploy(...)
. Two ways to fix it A) Use Git (recommended for reproducibility) - Tell Prefect where to pull the code from (your repo) and which file:function defines the flow.
Copy code
import os
import json
import sys
from prefect.flows import Flow
from prefect.docker.docker_image import DockerImage

def main():
    if len(sys.argv) != 2:
        print("Usage: python deploy.py <config_path>")
        sys.exit(1)

    config_path = sys.argv[1]
    with open(config_path, "r") as f:
        config = json.load(f)

    tags = config.get("tags", [])
    commit = os.getenv("GITHUB_SHA")
    if commit:
        tags.append(f"commit:{commit[:8]}")

    # Point to your Git repo and entrypoint (e.g., "flows/my_flow.py:my_flow")
    flow = Flow.from_source(
        source=config["git_url"],           # e.g., "<https://github.com/owner/repo.git>"
        entrypoint=config["entrypoint"],    # e.g., "flows/my_flow.py:my_flow"
    )

    docker_image = DockerImage(
        name="cyclopsai/template",
        dockerfile=config["docker_image"],  # path to your Dockerfile (or "auto")
    )

    flow.deploy(
        name=config["name"],
        work_pool_name=config["work_pool_name"],
        description=config.get("description", ""),
        tags=tags,
        push=True,
        image=docker_image,
        parameters=config.get("parameters", {}),
    )

if __name__ == "__main__":
    main()
Result: Prefect will add a
git_clone
pull step to each flow run so the worker can fetch your code. B) Bake the code into the image (no Git clone at runtime) - This works great, but you still need to tell Prefect where the code will be inside the container. - Use
Flow.from_source(".", entrypoint=...)
so Prefect records the relative path and sets the working directory at runtime. - Ensure your Dockerfile copies the project into the same path Prefect expects (by default
/opt/prefect/flows
if you use the “auto” builder). Example:
Copy code
import os
import json
import sys
from prefect.flows import Flow
from prefect.docker.docker_image import DockerImage

def main():
    if len(sys.argv) != 2:
        print("Usage: python deploy.py <config_path>")
        sys.exit(1)

    config_path = sys.argv[1]
    with open(config_path, "r") as f:
        config = json.load(f)

    tags = config.get("tags", [])
    commit = os.getenv("GITHUB_SHA")
    if commit:
        tags.append(f"commit:{commit[:8]}")

    # Record current directory as the source and the flow entrypoint
    flow = Flow.from_source(
        source=".",                         # local project root
        entrypoint=config["entrypoint"],    # e.g., "flows/my_flow.py:my_flow"
    )

    docker_image = DockerImage(
        name="cyclopsai/template",
        dockerfile=config["docker_image"],  # path to your Dockerfile OR "auto"
    )

    flow.deploy(
        name=config["name"],
        work_pool_name=config["work_pool_name"],
        description=config.get("description", ""),
        tags=tags,
        push=True,
        image=docker_image,
        parameters=config.get("parameters", {}),
        # optionally, ensure the worker's working dir matches your Dockerfile
        # job_variables={"working_dir": "/opt/prefect/flows"},
    )

if __name__ == "__main__":
    main()
If you use a custom Dockerfile, make sure it copies your code into
/opt/prefect/flows
(or set
job_variables["working_dir"]
to whatever path you use): ``` FROM prefecthq/prefect:3-python3.12
WORKDIR /opt/prefect/flows COPY . /opt/prefect/flows RUN pip install -r requirements.txt ``` Why your current script fails -
load_flow_from_entrypoint(...)
creates a Flow, but without
from_source(...)
Prefect does not have pull steps or a path to set at runtime. The deployment ends up with
storage=None
and
path=None
, leading to
/workspace/None
. Docs for reference - Store your flow code: Store flow code - Docker deployments: Docker deployments - Deployment concepts (pull steps, entrypoint, path): Deployment concepts If you can share: - Your
config["entrypoint"]
value - Whether you’re using a custom Dockerfile and its relevant lines (COPY/WORKDIR) - Which work pool/worker type you’re using …I can tailor the exact
from_source(...)
and Dockerfile/`job_variables` you should use.