We're trying to get a POC going on Prefect 2. We ...
# prefect-aws
s
We're trying to get a POC going on Prefect 2. We have our infra setup, but are having some issues running flows on task definitions. It seems like this is happening: • First run of a flow gets the following error
Copy code
prefect_aws.ecs.TaskFailedToStart: Cannotpullcontainererror: ref pull has been retried 1 time(s): failed to copy: httpReadSeeker: failed open: unexpected status code <https://registry-1.docker.io/v2/prefecthq/prefect/blobs/sha256:209b7d3b67f9dca78847c83ea6f0a78b0f91672ea1ccedc00e9bd5eb46315807>: 400 Bad Request
• 2nd and subsequent runs of the same flow gets a different error:
Copy code
Flow could not be retrieved from deployment. Traceback (most recent call last): File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/opt/prefect/flows/atlan.py", line 5, in <module> from util import SnowflakeDestination File "/opt/prefect/flows/util.py", line 3, in <module> import pandas as pd ModuleNotFoundError: No module named 'pandas' The above exception was the direct cause of the following exception: ScriptError: Script at 'flows/atlan.py' encountered an exception: ModuleNotFoundError("No module named 'pandas'")
• it appears that Prefect is creating a new task definition but it is not derived from our agent and therefore does not have the same dependencies installed. How can we resolve this?
c
Could you clarify what you mean by "derived from our agent"?
s
Thanks for clarifying. I meant derived from our agent's image. I got an answer from support@prefect.io on this last night. If I understand it correctly, anytime a block's Task definition parameters differ from the agent's task definition parameters Prefect spins up a separate task definition. This then means that the new task definition may not have the needed dependencies. Did I get that right?