Karl
05/19/2021, 12:41 AMnicholas
Karl
05/19/2021, 2:08 AM{
"config_overrides": {
"cloud": {
"agent": {
"auth_token": true
},
"auth_token": true
}
},
"env_vars": [],
"system_information": {
"platform": "Windows-10-10.0.18362-SP0",
"prefect_backend": "cloud",
"prefect_version": "0.14.19",
"python_version": "3.8.8"
}
}
I used the following guide to configure the docker storage and AWS roles. Many thanks to Jimmy Le!
https://www.lejimmy.com/distributed-data-pipelines-with-aws-ecs-fargate-and-prefect-cloud/
Even created a ecs-admin role as described in the former guide. With one exception, I created a AWS ECS Fargate
cluster in the AWS managment console. Which configured a default VPC ( the only VPC on my AWS account ).
First, I started my ECS Agent from my local machine Spyder IDE.
from prefect.agent.ecs.agent import ECSAgent
AGENT = ECSAgent(cluster='arn:aws:ecs:us-west-2:XXXXXXXXXXX:cluster/test-cluster',
#task_role_arn='arn:aws:iam::XXXXXXXXXXX:role/ecs-admin',
labels=['dev'])
AGENT.start()
The agent is recognized by the Prefect Cloud UI.
Then, from my local machine IDE I call script flow2.py.
-flow2py:
import prefect
from prefect.storage import Docker
from prefect.run_configs import ECSRun
from prefect import task, Flow
#import yaml
STORAGE = Docker(registry_url='<http://XXXXXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/|XXXXXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/>',
image_name='oceancluster-docker',
image_tag='latest',
dockerfile=r"C:\Users\XXXXXXXXXXX\cluster01-docker\Dockerfile.txt")
# definition = yaml.safe_load(
# """
# networkMode: awsvpc
# cpu: 512
# memory: 256
# containerDefinitions:
# - name: flow
# requiresCompatibilities:
# - FARGATE
# execution_role_arn: arn:aws:iam::XXXXXXXXXXX:role/ecsTaskExecutionRole
# """
# )
RUN_CONFIG = ECSRun(
run_task_kwargs = {
"taskDefinition":"14-fargate",
'cluster':'arn:aws:ecs:us-west-2:XXXXXXXXXXX:cluster/test-cluster',
'region_name':'us-west-2',
'networkConfiguration': {'awsvpcConfiguration': {'assignPublicIp': 'ENABLED', 'subnets': ['subnet-06eadc5f89d2293b2','subnet-0a2e199d91e06a92b'], 'securityGroups': []}},
'launchType':'FARGATE'
},
execution_role_arn='arn:aws:iam::XXXXXXXXXXX:role/ecs-admin',
#image='prefecthq/prefect:latest-python3.8',
memory='512',
cpu='256',
labels=['dev'])
#EXECUTOR = DaskExecutor()
@task
def say_hello():
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("Hello, Karl! Good Job, it worked@@!")
with Flow('hello-flow', storage=STORAGE, run_config=RUN_CONFIG ) as flow:
say_hello()
# Register the flow under the "tutorial" project
flow.register(project_name="test")
Which builds and pushes my docker image to AWS ECR. I can manually see the image when I log into my AWS ECR repository. Logging into the Prefect cloud, I see my flow and task. However, when I RUN my flow, I am presented with the following error message in my console and Prefect UI.
[2021-05-18 23:36:53,201] INFO - agent | Starting ECSAgent with labels ['dev']
[2021-05-18 23:36:53,202] INFO - agent | Agent documentation can be found at <https://docs.prefect.io/orchestration/>
[2021-05-18 23:36:53,204] INFO - agent | Agent connecting to the Prefect API at <https://api.prefect.io>
[2021-05-18 23:36:53,386] INFO - agent | Waiting for flow runs...
[2021-05-18 23:40:54,356] INFO - agent | Found 1 flow run(s) to submit for execution.
[2021-05-18 23:40:54,587] INFO - agent | Deploying flow run '5d5df04b-96b0-4128-bb73-e3c2b8a2ab0d'
[2021-05-18 23:40:54,833] ERROR - agent | Logging platform error for flow run 5d5df04b-96b0-4128-bb73-e3c2b8a2ab0d
[2021-05-18 23:40:55,163] ERROR - agent | Error while deploying flow
Traceback (most recent call last):
File "C:\Users\XXXXXXXXXXX\Anaconda3\envs\POC\lib\site-packages\prefect\agent\agent.py", line 420, in deploy_and_update_flow_run
deployment_info = self.deploy_flow(flow_run)
File "C:\Users\XXXXXXXXXXX\Anaconda3\envs\POC\lib\site-packages\prefect\agent\ecs\agent.py", line 297, in deploy_flow
resp = self.ecs_client.register_task_definition(**taskdef)
File "C:\Users\XXXXXXXXXXX\Anaconda3\envs\POC\lib\site-packages\botocore\client.py", line 386, in _api_call
return self._make_api_call(operation_name, kwargs)
File "C:\Users\XXXXXXXXXXX\Anaconda3\envs\POC\lib\site-packages\botocore\client.py", line 705, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.ClientException: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Role is not valid
Attached are screenshots of my AWS role*, VPC, and ECS Fargate configurations:
*appears that AWS no longer allows "AWSServiceRoleForECS" to be added to a role as illustrated in the guide.
{
"Stacks": [
{
"StackId": "arn:aws:cloudformation:us-west-2:XXXXXXXXXXX:stack/EC2ContainerService-test-cluster/69ab9fc0-b803-11eb-bcf3-06c940f8e023",
"StackName": "EC2ContainerService-test-cluster",
"Description": "AWS CloudFormation template to create a new VPC to work with Fargate in Create Cluster Wizard\n",
"Parameters": [
{
"ParameterKey": "SecurityIngressToPort",
"ParameterValue": "65535"
},
{
"ParameterKey": "EcsEndpoint",
"ParameterValue": ""
},
{
"ParameterKey": "IsFargate",
"ParameterValue": "true"
},
{
"ParameterKey": "IamRoleInstanceProfile",
"ParameterValue": ""
},
{
"ParameterKey": "EbsVolumeSize",
"ParameterValue": "0"
},
{
"ParameterKey": "SecurityIngressFromPort",
"ParameterValue": "0"
},
{
"ParameterKey": "SecurityGroupId",
"ParameterValue": ""
},
{
"ParameterKey": "EcsClusterName",
"ParameterValue": "test-cluster"
},
{
"ParameterKey": "SubnetIds",
"ParameterValue": ""
},
{
"ParameterKey": "KeyName",
"ParameterValue": ""
},
{
"ParameterKey": "SubnetCidr3",
"ParameterValue": ""
},
{
"ParameterKey": "SubnetCidr2",
"ParameterValue": "10.0.1.0/24"
},
{
"ParameterKey": "SubnetCidr1",
"ParameterValue": "10.0.0.0/24"
},
{
"ParameterKey": "SecurityIngressCidrIp",
"ParameterValue": "0.0.0.0/0"
},
{
"ParameterKey": "VpcId",
"ParameterValue": ""
},
{
"ParameterKey": "VpcCidr",
"ParameterValue": "10.0.0.0/16"
},
{
"ParameterKey": "AsgMaxSize",
"ParameterValue": "1"
},
{
"ParameterKey": "EcsInstanceType",
"ParameterValue": "m4.large"
},
{
"ParameterKey": "EbsVolumeType",
"ParameterValue": ""
},
{
"ParameterKey": "EcsAmiId",
"ParameterValue": "ami-084799b9fb64c149e"
},
{
"ParameterKey": "VpcAvailabilityZones",
"ParameterValue": "us-west-2a,us-west-2b,us-west-2c,us-west-2d"
},
{
"ParameterKey": "DeviceName",
"ParameterValue": ""
}
],
"CreationTime": "2021-05-18T18:03:56.292000+00:00",
"RollbackConfiguration": {},
"StackStatus": "CREATE_COMPLETE",
"DisableRollback": false,
"NotificationARNs": [],
"Outputs": [
{
"OutputKey": "TemplateVersion",
"OutputValue": "1.1.0",
"Description": "The version of the template used by Create Cluster Wizard"
},
{
"OutputKey": "UsedByECSCreateCluster",
"OutputValue": "true",
"Description": "Flag used by Elastic Container Service Create Cluster Wizard"
}
],
"Tags": [],
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
}
]
}
What am I missing ? Thank you in advance!!Kevin Kho