https://prefect.io logo
j

Jeremy Hetzel

08/25/2023, 4:56 PM
Update: Fixed! Description in the comments. I am seeing this error when deploying an ACI Job deployment:
Copy code
Response: {'detail': 'Error creating deployment: <ValidationError: "\'resource_group_name\' is a required property">'}
I created a worker-pool and worker:
Copy code
prefect work-pool create -t azure-container-instance azure2
prefect worker start -n my-aci-worker -p azure2
And then I have a deployment:
Copy code
# deployment.py

from os import environ, path
import sys
from prefect.deployments import Deployment
from prefect_azure.container_instance import AzureContainerInstanceJob
from azure.mgmt.containerinstance.models import ContainerGroupIdentity

azure_container_instance_job_block = AzureContainerInstanceJob.load("azure-test")

deployment = Deployment(
    name="azure-test",
    flow_name="get_repo_info",
    version="latest",
    description="Test deployment for Azure Container Instance.",
    infrastructure=azure_container_instance_job_block,
    work_pool_name="azure2",
    path="/opt/prefect/flows",
)

if __name__ == "__main__":
    deployment.apply()
When I run the deployment, I see an error:
Copy code
Response: {'detail': 'Error creating deployment: <ValidationError: "\'resource_group_name\' is a required property">'}
For more information check: <https://httpstatuses.com/409>
The
AzureContainerInstanceJob.load("azure-test")
has the correct resource_group_name:
Copy code
(Pdb) azure_container_instance_job_block.resource_group_name
'west-europe'
Do I need to specify a resource_group_name in the azure_container_instance worker? The worker is running on my local laptop. Does it need to be running in an Azure Container Instance instead?
Here is the full deployment trace:
Copy code
$ python deployment.py

Traceback (most recent call last):
  File "/home/jhetzel/src/c-core-labs/holmes/projects/azure/deployment2.py", line 31, in <module>
    deployment.apply()
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 255, in coroutine_wrapper
    return call()
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 382, in __call__
    return self.result()
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 282, in result
    return self.future.result(timeout=timeout)
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 168, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 345, in _run_async
    result = await coro
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/deployments/deployments.py", line 717, in apply
    deployment_id = await client.create_deployment(
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/client/orchestration.py", line 1479, in create_deployment
    response = await <http://self._client.post|self._client.post>(
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post
    return await self.request(
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1530, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/client/base.py", line 280, in send
    response.raise_for_status()
  File "/home/jhetzel/src/c-core-labs/holmes/.venv/lib/python3.10/site-packages/prefect/client/base.py", line 138, in raise_for_status
    raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
prefect.exceptions.PrefectHTTPStatusError: Client error '409 Conflict' for url '<https://api.prefect.cloud/api/accounts/16005bcf-76b3-4ba3-867f-e6f0b3faba97/workspaces/168ac782-a029-429a-968e-e14a6c0f6f06/deployments/>'
Response: {'detail': 'Error creating deployment: <ValidationError: "\'resource_group_name\' is a required property">'}
For more information check: <https://httpstatuses.com/409>
If I change the
worker_pool_name
to a an Azure push worker pool, the deployment succeeds.
Fixed! We had to add the resource group name, identities, and ACRManagedIdentity to the created worker pool via the web interface.
I thought defining these in the ACI block used by the deployment would have sufficed, but easy enough to duplicate it in the worker pool configuration.