Hi, I need to run a fargate task in aws. I have fo...
# prefect-community
j
Hi, I need to run a fargate task in aws. I have followed this tutorial (https://docs.prefect.io/orchestration/execution/fargate_task_environment.html#examples) When I run this flow via flow.run(), the stdout print "Flow run SUCCESS: all reference tasks succeeded", the task does not run. What am I supposed to execute this flow?
d
Hi @jamodes!
flow.run()
on your local machine is intended to test the flow locally and will not submit your Flow Run to Fargate. For that you’ll need a Fargate Agent. Once you register the Flow with Prefect Cloud your scheduled runs will be submitted to Fargate by the agent 👍
j
thank you @Dylan, do we have to register to Perfect cloud? Can't we do this in local machine?
d
Since Prefect Flows can run in many environments and most users wish to run their workflow on a schedule or in response to an event (which can make an API call to Prefect Cloud), we’ve designed the Agents as our mechanism to submit work to cloud environments
You can write a Flow that runs locally and submits tasks to Fargate using the AWS python clients
Some AWS Tasks are already in our Task Library: https://docs.prefect.io/api/latest/tasks/aws.html
But it looks like no one has contributed a
SubmitFargateTask
just yet
j
Thanks a lot. I just want to understand whether we are able to deploy our flows to our proprietary server or not.
d
What do you mean by proprietary here? In a private cloud?
m
@jamodes it should be no problem starting Fargate runs from on premise. Just use flow.register and a fargate agent. The fargate agent will pick up the flow and run it. Without Prefect cloud
I had issues in the past with FargateTaskEnvironment and used a normal RemoteEnvironment. The FargateAgent was used and got all the Fargate related configs