https://prefect.io logo
p

psimakis

11/18/2020, 3:40 PM
Hello everyone, I'm trying to configure a fargate agent but I probably miss something. Prefect server is running locally and I run a fargate agent using the following command:
Copy code
$ prefect agent fargate start cpu=256 memory=512 executionRoleArn=arn:aws:iam::xxxxxxxxx:role/ecsTaskExecutionRole networkConfiguration="{'awsvpcConfiguration':{'assignPublicIp': 'ENABLED', 'subnets': ['subnet-xxxxxx'], 'securityGroups': ['sg-xxxxxxxxxxx']}}" cluster=fargate-cluster
Below you will find the flow:
Copy code
from prefect import Flow, task, Parameter
from prefect.environments.storage import Docker


@task
def square_int(x):
    return x * x

@task
def sub_2(x):
    return x - 2


with Flow(
    name='sp-flow-test',
    storage=Docker(registry_url='xxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com',
                   image_name='prefecttest', image_tag='latest'),

) as flow:
    integer = Parameter('integer', default=2)
    x_sqr = square_int(integer)
    x_less_2 = sub_2(x_sqr)

flow.register(project_name='test', idempotency_key=flow.serialized_hash())
When I run the flow, the fargate task seems to run fine (check out the screenshot). The problem is that flow run stays
Submitted
forever. Have you any clue? Thanks.
s

Spencer

11/18/2020, 3:43 PM
I'd suggest looking at the logs of those tasks.
p

psimakis

11/18/2020, 3:58 PM
@Spencer good point! I will pass
logConfiguration
and I will come back with more details
fargate is trying to connect to localhost 😕 :
Copy code
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: /graphql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7cec8e1250>: Failed to establish a new connection: [Errno 111] Connection refused'))
Should I expose prefect server to www?
s

Spencer

11/18/2020, 6:18 PM
Well, these tasks need to have the API endpoint configured in the environment. I suppose you can look at the task definition for those tasks and verify the
PREFECT__CLOUD__API
environment variable?
p

psimakis

11/19/2020, 7:26 AM
Exposing the server could solve this issue even if I'm not using prefect cloud?
s

Spencer

11/19/2020, 1:04 PM
Either way a server needs to be accessible from the task; it doesn't appear that
localhost
is running prefect server. 🙂
👍 1