https://prefect.io logo
Powered by
Title
a

Abhinav Chordia

04/18/2023, 3:53 PM
What is the recommended way of building custom images for flows? More details in thread 🧵
I’m launching my flow like this: 
Deployment.build_from_flow(
        flow=flow,
        name=name,
        work_queue_name=work_queue,
      path=f"/user/{os.getlogin()}/flow/{flow.__name__}/deployments/{name}",
        infrastructure=get_infra_block(infra_block),
        storage=get_storage_block(storage_block),
        infra_overrides={
            "env": {
                "PREFECT_LOGGING_LEVEL": "DEBUG",
                # "EXTRA_PIP_PACKAGES":"adlfs"
            },
            "image" : "<http://docker.gh.st/hypo:dev|docker.gh.st/hypo:dev>"
        },
    )
where the image is built like this: 
# syntax=docker/dockerfile:1
FROM --platform=linux/amd64 prefecthq/prefect:2-python3.9-kubernetes

COPY pyproject.toml .

RUN pip install --no-cache-dir --upgrade pip && pip install pip-tools && python -m piptools compile --resolver=backtracking pyproject.toml && pip install -r requirements.txt
When I try to run the flow, I get the following error: 
Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "<frozen importlib._bootstrap_external>", line 846, in exec_module
  File "<frozen importlib._bootstrap_external>", line 982, in get_code
  File "<frozen importlib._bootstrap_external>", line 1039, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'src/hypo/flows/example/example_flow.py'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 276, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/usr/local/lib/python3.9/site-packages/prefect/client/utilities.py", line 40, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/prefect/deployments.py", line 217, in load_flow_from_flow_run
    flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.9/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.9/site-packages/prefect/flows.py", line 809, in load_flow_from_entrypoint
    flow = import_object(entrypoint)
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/importtools.py", line 201, in import_object
    module = load_script_as_module(script_path)
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/importtools.py", line 164, in load_script_as_module
    raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'src/hypo/flows/example/example_flow.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')
Screen Shot 2023-04-18 at 11.55.43 AM.png
I’ve confirmed that the remote azure storage has the uploaded code.
And everything works fine if I change the deployment to be this: 
Deployment.build_from_flow(
        flow=flow,
        name=name,
        work_queue_name=work_queue,
        path=f"/user/{os.getlogin()}/flow/{flow.__name__}/deployments/{name}",
        infrastructure=get_infra_block(infra_block),
        storage=get_storage_block(storage_block),
        infra_overrides={
            "env": {
                "PREFECT_LOGGING_LEVEL": "DEBUG",
                "EXTRA_PIP_PACKAGES":"adlfs"
            },
        },
    )
Where I’m just using the default prefect image.
Is there a recommendation on how to build custom images so that we can pre-install required deps?
I also understand that I could copy the local flows directly into the Docker image but the user workflows would then require every user to push their own image rather than just uploading the code to remote storage and have it pulled down into the image.
r

Ryan Peden

04/18/2023, 8:15 PM
The default Prefect image uses this Bash script as its entrypoint. It installs any packages listed in 
EXTRA_PIP_PACKAGES
before running the deployment. Here's the Dockerfile Prefect uses for this. You'll find the entrypoint right down at the bottom. Perhaps using this script or something similar would help. I see you're already installing dependencies from a requirements.txt, though. If 
adlfs
is the only thing missing, you could also just install it when building your container image so you won't need to use 
EXTRA_PIP_PACKAGES
.
a

Abhinav Chordia

04/18/2023, 9:31 PM
The script isn’t the problem IMO. I’m trying to install dependencies on top of the prefect image and use that for my k8s jobs but for some reason when I do that, prefect can’t find my flows once it’s downloaded from remote storage.
r

Ryan Peden

04/18/2023, 9:34 PM
Ahh - got it. My apologies, I misread a couple of things above.
a

Abhinav Chordia

04/18/2023, 9:36 PM
no worries. Thank you for trying to help!
#prefect-getting-startedJoin Slack
Powered by