https://prefect.io logo
a

Alberto de Santos

10/15/2020, 4:15 PM
Hi community! Can anyone share a (best-practice) script to setup Agents, Flows and create_flow_run? Not the standard, but just a prod-like script, allowing for instance to setup an agent if down, ensure the agnet is working properly and so on…
d

Dylan

10/15/2020, 4:17 PM
Hi @Alberto de Santos! What environment are you looking to set your agent up in?
Can you describe the behavior you’re seeing in a little more detail? Are your flow runs moving into a
Submitted
state?
a

Alberto de Santos

10/15/2020, 4:21 PM
Ok, let’s start with the flows first
So, I am setting up a schedule for them in this way:
Copy code
schedule = Schedule(clocks=[DatesClock([pendulum.instance(self.get_start_time())])])
where get_start_time takes a parameter from here : {‘fecha’:‘15/10/2020’, ‘hora’‘1612’}
fecha/hora means date/hour
d

Dylan

10/15/2020, 4:23 PM
right 😄
I need to AFK for a few minutes but I will be right back
But, I think you might want to check out
a

Alberto de Santos

10/15/2020, 4:24 PM
then, processes are queued as usual, in this way:
d

Dylan

10/15/2020, 4:24 PM
What’s your overall goal for the flow’s schedule? To run every minute, every hour?
a

Alberto de Santos

10/15/2020, 4:24 PM
run it at a specific time, actually
I would like a task/process to be run EXACTLY at a specific moment
kind of a trigger
In fact, when I do the previous thing and I add elements to the queue, it only keeps one element in the queue and all the rest tasks (despite of being scheduled for a specific time) they are run immediately
I think the option
scheduled_start_time
in
create_flow_run
can do the job as I desire it 🙂
Thanks guys
Concerning he script to setup agents? Do you have any piece of advice?
d

Dylan

10/15/2020, 4:50 PM
What determines the time the flow needs to run at?
This outlines how to set up a supervised agent. Supervisor should restart the process if it dies, etc
a

Alberto de Santos

10/15/2020, 4:51 PM
Ah ok!
Well it actually doesn’t die, it is alive but not taking more process, no idea why
But thanks
d

Dylan

10/15/2020, 4:52 PM
Are you running with Prefect Cloud? If so, you may want to check on your concurrency slots
a

Alberto de Santos

10/15/2020, 4:52 PM
yep, only one slot
but in any case, it should deal with all of them, one by one, I know, but should deal with them
d

Dylan

10/15/2020, 4:53 PM
A flow run should move from
Scheduled
->
Submitted
->
Running
When it moves from
Scheduled
->
Submitted
it means that an agent has picked up the flow run and is creating the job/process necessary for the flow run to happen
When you’re experiencing this issue, it looks like the Flow Run isn’t moving to a
Submitted
state
Which means your agent may be dying / running out of resources
What logs are you seeing?
a

Alberto de Santos

10/15/2020, 4:55 PM
it says sth like waiting
They are never submitted (only with those yellow/palid yellow bars)
like this
and then, they move from that state to a yellow solid one, with a clock on top
blue color doesn’t appear at all
d

Dylan

10/15/2020, 5:07 PM
Do you have a flow run id for one of these runs?
I’d take a look at the logs
The yellow state is
Submitted
So it sounds like the agent is submitting the run
But something is happening when it tries to start
OH
You’re saying it’s never submitted?
That’s a problem on the agent side
a

Alberto de Santos

10/15/2020, 5:43 PM
look, here again
my agent id is 04548e2f-83a4-4e87-8891-8c7913287ffc
and here it is
why is it in that status?
only 26 minutes ago?
What if I run things daily?
d

Dylan

10/15/2020, 6:31 PM
Ah see
@Alberto de Santos That means your agent isn’t connected to prefect cloud
a

Alberto de Santos

10/15/2020, 8:18 PM
uhm! after exploration it is true (it was a bit confusing to me, as it doesn’t say so literally), but yes, it was a matter of using
nohup