https://prefect.io
Join Slack
Hi there, I'm new to Prefect and checking out options to see if Prefect will work for my company. I'...
# ask-community
c
Hi there, I'm new to Prefect and checking out options to see if Prefect will work for my company. I'm having trouble starting an ECS Agent within our infrastructure that communicates with PrefectCloud. We are using HIPAA compliant AWS architecture as described here, i.e. most of the infrastructure is in private VPCs and exposed via a public VPC. Can you help me figure out if that's my issue?
I ran the following start command from within a private vpc:
Copy code
prefect agent ecs start \
    --cluster <arn> \
    --run-task-kwargs "config.yaml"
where 
config.yaml
contains:
Copy code
networkConfiguration:
  awsvpcConfiguration:
    assignPublicIP: ENABLE
    subnets: [private_subnet]
    securityGroups: [security_group]
This is the (abridged) connection refused error I'm getting:
Copy code
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa5b7613ed0>: Failed to establish a new connection: [Errno 111] Connection refused'))
k
Hi @Claire Herdeman, do you mean the Agent is even able to be seen by Cloud?
c
No, it doesn't seem to launch at all
I got a similar connection error when I ran 
prefect create project <name>
k
Is this ecs agent running on an EC2? Does it have permissions for HTTP going out?
c
No, i'm just trying to run 
prefect ecs start
also this is my first time trying to get it set up so I could be misunderstanding agent usage!
k
Let me try setting up an ec2 on my end without traffic allowed and see if I replicate
🙏 1
c
As another workaround, would using server rather than cloud be easier if it can be launched within our infrastructure?
k
Are you on Prefect Cloud or Server?
c
I just set up a free Prefect Cloud
k
We do have people using Prefect Server because of regulations that don’t let them have connections, but really Prefect is a tool of choice for organizations that need HIPAA compliant because we don’t see any of the data or the code
c
Ahhh makes sense
k
Could you walk me over what you did? Did you change your backend to server by any chance? Did you authenticate Prefect with the CLI?
c
I actually did change backend to server, how do I switch back? and I'm not sure, which means probably not. What's the best way to do that?
k
prefect backend cloud
to change the backend first and then try starting the agent again?
c
looks like that did it, thank you so much for your help!
k
Oh nice! Happy to help 🙂
That error was because it was trying to access the local database spun up by Prefect server at localhost:4200, but that wasn’t available.
67 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by