:azure: :docker_ship: Docker infrastructure block ...
# ask-community
s
azure docker ship Docker infrastructure block tries to access flow file from local system instead of azure storage Hi all, I am currently running a prefect cloud, with azure blob storage as storage block and a docker container from a private repository as an infrastructure block. The issue that I am facing when running the flow. The agent that I am running on my local machine pulls the docker image and runs it (confirmed from the container logs in docker desktop) before throwing the following error and exiting:
Copy code
03 13:24:50   File "/usr/local/lib/python3.7/site-packages/prefect/engine.py", line 246, in retrieve_flow_then_begin_flow_run
2023-03-03 13:24:50     flow = await load_flow_from_flow_run(flow_run, client=client)
2023-03-03 13:24:50   File "/usr/local/lib/python3.7/site-packages/prefect/client.py", line 105, in with_injected_client
2023-03-03 13:24:50     return await fn(*args, **kwargs)
2023-03-03 13:24:50   File "/usr/local/lib/python3.7/site-packages/prefect/deployments.py", line 73, in load_flow_from_flow_run
2023-03-03 13:24:50     flow = await run_sync_in_worker_thread(import_object, str(import_path))
2023-03-03 13:24:50   File "/usr/local/lib/python3.7/site-packages/prefect/utilities/asyncutils.py", line 56, in run_sync_in_worker_thread
2023-03-03 13:24:50     return await anyio.to_thread.run_sync(call, cancellable=True)
2023-03-03 13:24:50   File "/usr/local/lib/python3.7/site-packages/anyio/to_thread.py", line 32, in run_sync
2023-03-03 13:24:50     func, *args, cancellable=cancellable, limiter=limiter
2023-03-03 13:24:50   File "/usr/local/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
2023-03-03 13:24:50     return await future
2023-03-03 13:24:50   File "/usr/local/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 867, in run
2023-03-03 13:24:50     result = context.run(func, *args)
2023-03-03 13:24:50   File "/usr/local/lib/python3.7/site-packages/prefect/utilities/importtools.py", line 193, in import_object
2023-03-03 13:24:50     module = load_script_as_module(script_path)
2023-03-03 13:24:50   File "/usr/local/lib/python3.7/site-packages/prefect/utilities/importtools.py", line 156, in load_script_as_module
2023-03-03 13:24:50     raise ScriptError(user_exc=exc, path=path) from exc
2023-03-03 13:24:50 prefect.exceptions.ScriptError: Script at '<Flow script location on local machine>' encountered an exception
This seems related to this issue however as I am running prefect 2.1.0
DockerFlowRunner
has been deprecated. I tried the
COPY $FLOWSCRIPT.py .
before building the docker image as well but it didn't help, and isn't a solution that will work for me. The link in there to the s3 with volume parameters for the credentials also doesn't seem relevant, as I have already loaded and authorised the storage block using SAS. The same issue occurs even when I deploy using the CLI according to the prompt in the second to last row of this table. Could someone please help me to get the docker container to look for the flow script in the storage block where it is successfully uploaded on deployment, not the path of the flow script on the machine that initiates the deployment?
1
For context, I am using a setup whose infra blocks are identical to that described by Anna here and the flow is just a print function instead of a logger.
c
Hi Sebastian, you are using Prefect
2.1.0
?
s
Hi Christopher, I am.
c
I would highly, highly recommend updating before going down a troubleshooting path here, as there have been numerous changes on resiliency, infrastructure, blocks, etc. The latest is
2.8.4
- https://github.com/PrefectHQ/prefect/releases
s
Thanks, I don't think that is a possibility due to a delicate dependency situation in the main image. Would you suggest running the agent on 2.8.4 if the docker container code is still on 2.1.0?
c
You’ll get attribute errors when running your code , so I don’t recommend it
there are a number of changes (many which would be breaking) between the two; DeploymentSpec is deprecated in favor of deployment objects, there have been numerous fixes and additions to fields / flags (such as path and entrypoints for your docker container) etc.
👍 1