Hi all, I'm having some trouble getting prefect c...
# ask-community
s
Hi all, I'm having some trouble getting prefect cloud to work with azure blob storage and docker infrastructure. The docker image, the environment I deploy from, and the agent environment are all identical with prefect 2.8.4 My deployment script looks as follows:
Copy code
from prefect.infrastructure.docker import DockerContainer
from prefect.filesystems import Azure
from prefect.deployments import Deployment
from prefect import flow 

az_block = Azure.load("$AzureContainerBlock")
usecase_container = DockerContainer.load("$DockerBlock")

@flow
def abc_train():
    print("abc_train_internal")


docker_deploy = Deployment.build_from_flow(
            flow=abc_train,
            name="abc_train",
            work_queue_name="test",
            version="18",
            storage=az_block,
            path='abc',
            infrastructure=usecase_container,
            infra_overrides={"env": {"PREFECT_LOGGING_LEVEL": "DEBUG"}}
            )

if __name__ == "__main__":
    docker_deploy.apply()
When I deploy, the upload to azure blob storage happens as expected. However, when I run the flow (agent running on local machine) I get the following logs:
Copy code
---INFO---
Downloading flow code from storage at 'abc'
---ERROR---
Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/prefect/engine.py", line 274, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/usr/local/lib/python3.7/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/prefect/deployments.py", line 175, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/usr/local/lib/python3.7/site-packages/prefect/filesystems.py", line 708, in get_directory
    from_path=from_path, local_path=local_path
  File "/usr/local/lib/python3.7/site-packages/prefect/filesystems.py", line 322, in get_directory
    return self.filesystem.get(from_path, local_path, recursive=True)
  File "/usr/local/lib/python3.7/site-packages/fsspec/asyn.py", line 293, in get
    [os.makedirs(os.path.dirname(lp), exist_ok=True) for lp in lpaths]
  File "/usr/local/lib/python3.7/site-packages/fsspec/asyn.py", line 293, in <listcomp>
    [os.makedirs(os.path.dirname(lp), exist_ok=True) for lp in lpaths]
  File "/usr/local/lib/python3.7/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: ''
When going line by line through
retrieve_flow_then_begin_flow_run
in an interative python session everything seems to be truly fine, until the
storage_block.get_directory()
call. Interactively,
deployment.path
is
'<az://testing-ib>'
which is the correct container name per the storage block. Would anyone be able to help me debug this one?
c
HI Sebastian, Can you add
entrypoint=<file_name.py>:abc_train
to your deploy arguments? replace file name with the name of your deployment.py file you have listed
s
Hi Christopher, just retried that and still the same issue. The entrypoint should be inferred by
build_from_flow
right?
c
it should yes - I’ve had interesting behavior when I haven’t been able to provide the actual flow object. What is the full path structure from
/
for your flow?