https://prefect.io
Join Slack
Hello everyone, I have my prefect flow py file inside a virtual environment in a remote ec2 instance...
# ask-community
h
Hello everyone, I have my prefect flow py file inside a virtual environment in a remote ec2 instance how do I run the file such that when I log out it will remember and pick up the schedule indicated
j
Hi @Hedgar, are you using prefect cloud or prefect server to orchestrate the flow runs? Are you running a prefect agent in the EC2 instance?
upvote 1
a
To run flows on schedule, you need to have an agent and register a flow - check out this docs page https://docs.prefect.io/orchestration/getting-started/registering-and-running-a-flow.html#register-a-flow
h
I'm using perfect sever. For my case which agent would fit a remote ec2 instance.. Local?
@jcozar prefect sever
a
Yes, both local and Docker agent would use the resources on that EC2 machine. But note that you can start any agent there - agent is only polling for new flow runs. The actual execution may be somewhere else, e.g. in a Kubernetes cluster on EKS
h
@Anna Geller I'm trying to keep it simple as much as possible. I'm trying perfect because crontab wasn't meeting my expectations especially in the area of retry when failure occur
👍 1
@Anna Geller after creating an agent and registering the flow, if my py file is within a virtual environment that needs to be activated before the code-py file can run do I need to do any extra? I also know that while registering the flow I'm suppose to indicate the path to the py file, would this process involve activating the virtual environment?
a
I assume you’re talking about a local agent. You can attach a path when starting an agent and you can also start the agent in a local environment, this way your flows deployed by this agent will have all the packages installed in that environment. Here is an example config file for local agent with Conda and supervisor: https://prefect-community.slack.com/archives/CL09KU1K7/p1645700047652749?thread_ts=1645699804.651089&cid=CL09KU1K7
prefect agent local start -p /Users/you/your/module/
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by