Channels
#prefect-community
Title
a
Alberto de Santos
10/15/2020, 4:15 PMHi community!
Can anyone share a (best-practice) script to setup Agents, Flows and create_flow_run? Not the standard, but just a prod-like script, allowing for instance to setup an agent if down, ensure the agnet is working properly and so on…
d
Dylan
10/15/2020, 4:17 PM
Hi @Alberto de Santos!
What environment are you looking to set your agent up in?
You might want to look at: https://docs.prefect.io/orchestration/tutorial/multiple.html#install-a-supervised-agent
Can you describe the behavior you’re seeing in a little more detail? Are your flow runs moving into a
Submitteda
Alberto de Santos
10/15/2020, 4:21 PMOk, let’s start with the flows first
So, I am setting up a schedule for them in this way:
Copy code
schedule = Schedule(clocks=[DatesClock([pendulum.instance(self.get_start_time())])])where get_start_time takes a parameter from here : {‘fecha’:‘15/10/2020’, ‘hora’‘1612’}
fecha/hora means date/hour
d
Dylan
10/15/2020, 4:23 PM
right 😄
I need to AFK for a few minutes but I will be right back
But, I think you might want to check out
a
Alberto de Santos
10/15/2020, 4:24 PMthen, processes are queued as usual, in this way:
d
Dylan
10/15/2020, 4:24 PM
What’s your overall goal for the flow’s schedule? To run every minute, every hour?
a
Alberto de Santos
10/15/2020, 4:24 PMrun it at a specific time, actually
I would like a task/process to be run EXACTLY at a specific moment
kind of a trigger
In fact, when I do the previous thing and I add elements to the queue, it only keeps one element in the queue and all the rest tasks (despite of being scheduled for a specific time) they are run immediately
I think the option
scheduled_start_timecreate_flow_runThanks guys
Concerning he script to setup agents? Do you have any piece of advice?
d
Dylan
10/15/2020, 4:50 PM
What determines the time the flow needs to run at?
This outlines how to set up a supervised agent. Supervisor should restart the process if it dies, etc
a
Alberto de Santos
10/15/2020, 4:51 PMAh ok!
Well it actually doesn’t die, it is alive but not taking more process, no idea why
But thanks
d
Dylan
10/15/2020, 4:52 PM
Are you running with Prefect Cloud? If so, you may want to check on your concurrency slots
a
Alberto de Santos
10/15/2020, 4:52 PMyep, only one slot
but in any case, it should deal with all of them, one by one, I know, but should deal with them
d
Dylan
10/15/2020, 4:53 PM
A flow run should move from
ScheduledSubmittedRunningWhen it moves from
ScheduledSubmittedWhen you’re experiencing this issue, it looks like the Flow Run isn’t moving to a
SubmittedWhich means your agent may be dying / running out of resources
What logs are you seeing?
a
Alberto de Santos
10/15/2020, 4:55 PMit says sth like waiting
They are never submitted (only with those yellow/palid yellow bars)
like this
and then, they move from that state to a yellow solid one, with a clock on top
blue color doesn’t appear at all
d
Dylan
10/15/2020, 5:07 PM
Do you have a flow run id for one of these runs?
I’d take a look at the logs
The yellow state is
SubmittedSo it sounds like the agent is submitting the run
But something is happening when it tries to start
OH
You’re saying it’s never submitted?
That’s a problem on the agent side
a
Alberto de Santos
10/15/2020, 5:43 PMlook, here again
my agent id is 04548e2f-83a4-4e87-8891-8c7913287ffc
and here it is
why is it in that status?
only 26 minutes ago?
What if I run things daily?
d
Dylan
10/15/2020, 6:31 PM
Ah see
@Alberto de Santos That means your agent isn’t connected to prefect cloud
a
Alberto de Santos
10/15/2020, 8:18 PMuhm! after exploration it is true (it was a bit confusing to me, as it doesn’t say so literally), but yes, it was a matter of using
nohup