Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
c
Claire Herdeman
05/26/2021, 9:02 PMHi there, I'm new to Prefect and checking out options to see if Prefect will work for my company. I'm having trouble starting an ECS Agent within our infrastructure that communicates with PrefectCloud. We are using HIPAA compliant AWS architecture as described here, i.e. most of the infrastructure is in private VPCs and exposed via a public VPC. Can you help me figure out if that's my issue?
I ran the following start command from within a private vpc:
prefect agent ecs start \
--cluster <arn> \
--run-task-kwargs "config.yaml"where
config.yamlnetworkConfiguration:
awsvpcConfiguration:
assignPublicIP: ENABLE
subnets: [private_subnet]
securityGroups: [security_group]This is the (abridged) connection refused error I'm getting:
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa5b7613ed0>: Failed to establish a new connection: [Errno 111] Connection refused'))k
Kevin Kho
05/26/2021, 9:06 PMHi @Claire Herdeman, do you mean the Agent is even able to be seen by Cloud?
c
Claire Herdeman
05/26/2021, 9:07 PMNo, it doesn't seem to launch at all
I got a similar connection error when I ran
prefect create project <name>k
Kevin Kho
05/26/2021, 9:08 PMIs this ecs agent running on an EC2? Does it have permissions for HTTP going out?
c
Claire Herdeman
05/26/2021, 9:10 PMNo, i'm just trying to run
prefect ecs startalso this is my first time trying to get it set up so I could be misunderstanding agent usage!
k
Kevin Kho
05/26/2021, 9:14 PMLet me try setting up an ec2 on my end without traffic allowed and see if I replicate
🙏 1
Are you on Prefect Cloud or Server?
c
Claire Herdeman
05/26/2021, 9:15 PMAs another workaround, would using server rather than cloud be easier if it can be launched within our infrastructure?
I just set up a free Prefect Cloud
k
Kevin Kho
05/26/2021, 9:16 PMWe do have people using Prefect Server because of regulations that don’t let them have connections, but really Prefect is a tool of choice for organizations that need HIPAA compliant because we don’t see any of the data or the code
c
Claire Herdeman
05/26/2021, 9:19 PMAhhh makes sense
k
Kevin Kho
05/26/2021, 9:38 PMCould you walk me over what you did? Did you change your backend to server by any chance? Did you authenticate Prefect with the CLI?
c
Claire Herdeman
05/26/2021, 9:39 PMI actually did change backend to server, how do I switch back? and I'm not sure, which means probably not. What's the best way to do that?
k
Kevin Kho
05/26/2021, 9:40 PMprefect backend cloudc
Claire Herdeman
05/26/2021, 9:42 PMlooks like that did it, thank you so much for your help!
k
Kevin Kho
05/26/2021, 9:42 PMOh nice! Happy to help 🙂
That error was because it was trying to access the local database spun up by Prefect server at localhost:4200, but that wasn’t available.