https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> What might be causing a `Flow could n...
# ask-marvin
m
@Marvin What might be causing a 
Flow could not be submitted due to infrastructure.
error?
m
Oh, the joys of troubleshooting. The error 
Flow could not be submitted due to infrastructure
typically indicates an issue with the environment in which your flow is trying to execute. Prefect uses infrastructure to create the environment for a user's flow to execute. This infrastructure is attached to a deployment and is propagated to flow runs created for that deployment. The infrastructure has two main jobs: 1. Create execution environment infrastructure for the flow run. 2. Run a Python command to start the prefect engine in the infrastructure, which retrieves the flow from storage and executes the flow. The infrastructure types Prefect currently provides include: - Process: runs flows in a local subprocess. - DockerContainer: runs flows in a Docker container. - KubernetesJob: runs flows in a Kubernetes Job. - ECSTask: runs flows in an Amazon ECS Task. If you're running the flow directly by calling it yourself, remember that you are responsible for the environment in which the flow executes. You might want to check the specifics of your infrastructure setup. For more information, you can visit the Prefect documentation here and here. But then again, what do I know? I'm just a robot.
n
typically this is a permissions error in the execution layer of some kind, or sometimes an unhandled failure to retrieve flow code do you have the worker logs from this time?
👍 1
m
@Nate I resized the Agent, I think it's good now?
n
hmm - hard to say without agent logs but if its working now then 👍
m
@Nate Ack, it popped up again. I'm not sure how to drill deeper - it just says
Copy code
Encountered exception during execution:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/prefect/engine.py", line 1729, in orchestrate_task_run
    result = await call.aresult()
  File "/opt/conda/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 291, in aresult
    return await asyncio.wrap_future(self.future)
  File "/opt/conda/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 315, in _run_sync
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/srm-data-science/nba_feature_engineering.py", line 396, in skippable_deployment_task
    raise Exception(flow_state.message)
Exception: Flow run could not be submitted to infrastructure
Finished in state Failed('Task run encountered an exception Exception: Flow run could not be submitted to infrastructure')
when I look at the Subflow logs, it just says it's in a 
Crashed
state, has no logs, and under 
Details
has
Copy code
State Message
Flow run could not be submitted to infrastructure
n
you'll want to check out the agent logs i think
👍 1
m
aha, thanks!
@Nate Not sure how useful any of this is
Copy code
|`18:23:49.235 \| ERROR \| prefect.agent - Failed to submit flow run '6ab5bded-44c8-4fb2-bde3-9fa76fa97364' to infrastructure.`|
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/prefect/agent.py", line 500, in _submit_run_and_capture_errors`|
result = await infrastructure.run(task_status=task_status)
File "/opt/conda/lib/python3.10/site-packages/prefect_aws/ecs.py", line 621, in run
) = await run_sync_in_worker_thread(`
File "/opt/conda/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
return await anyio.to_thread.run_sync(	
File "/opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread	
return await future	
File "/opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run	
result = context.run(func, *args)	
File "/opt/conda/lib/python3.10/site-packages/prefect_aws/ecs.py", line 814, in _create_task_and_wait_for_start	
self._report_task_run_creation_failure(task_run, exc)	
File "/opt/conda/lib/python3.10/site-packages/prefect_aws/ecs.py", line 810, in _create_task_and_wait_for_start	
task = self._run_task(ecs_client, task_run)	
File "/opt/conda/lib/python3.10/site-packages/prefect_aws/ecs.py", line 1487, in _run_task	
return ecs_client.run_task(**task_run)["tasks"][0]	
IndexError: list index out of range
66 Views
Open in Slack
PreviousNext
Powered by