https://prefect.io logo
#prefect-community
Title
# prefect-community
j

Jared

06/01/2020, 6:52 PM
Testing out Prefect as an alternative for an enterprise deployment and running into an odd issue with scheduled flows. For some reason I can only get
IntervalSchedule
tasks to run 10 times (this is using Core/not on cloud). Am I missing something obvious? When I initially register the flow, 10 runs get queued and executed, but no more. In the terminal running the server, the scheduler wakes, schedules 0 runs, and sleeps even if all 10 original runs have passed. On-demand runs in the UI still work at this point. I'll comment with what I'm doing to reproduce.
Using Prefect 0.11.4 in a Conda env on Windows 10.
prefect server start
and
prefect agent start
both run fine. Then from the interpreter in the same env:
Copy code
from datetime import timedelta, datetime
from prefect.schedules import IntervalSchedule

@task
def getone():
    return 1

schedule = IntervalSchedule(
    start_date=datetime.utcnow() + timedelta(seconds=1),
    interval=timedelta(minutes=1),
)

with Flow("testflow", schedule=schedule) as flow:
    getone()

flow.register()
k

Kyle Moon-Wright

06/01/2020, 7:15 PM
Hey @Jared! I’ve been able to reproduce your flow and just wanted to clarify that the scheduler schedules 10 runs out, and should continue to auto-populate future runs after those. Are you not seeing new runs populating Upcoming Runs?
j

Jared

06/01/2020, 8:00 PM
@Kyle Moon-Wright that's correct, new runs aren't coming into Upcoming Runs after the initial 10. I can even see that the scheduler is running, it's just not picking up this flow:
Copy code
scheduler_1  | [2020-05-31 12:10:53,195] INFO - prefect-server.Scheduler | Scheduled 0 flow runs.
graphql_1    | INFO:     192.168.0.6:55604 - "POST /graphql/ HTTP/1.1" 200 OK
scheduler_1  | [2020-05-31 12:10:53,296] DEBUG - prefect-server.Scheduler | Sleeping for 300.0 seconds...
k

Kyle Moon-Wright

06/01/2020, 8:37 PM
Very strange, I am not able to fully reproduce your issue. I’ve registered your
testflow
on my own Prefect Server (using 0.11.4) with a Local Agent and after 5 flow runs, the UI repopulates the queue to 10 Upcoming Runs on my Dashboard. Would you mind opening issue on Github for greater visibility for the team?
FWIW, this is what I’m seeing when the next 5 runs are scheduled:
Copy code
scheduler_1  | [2020-06-01 20:43:28,833] DEBUG - prefect-server.api.schedules | Flow run <flow_run_id> of flow <flow_id> scheduled for 2020-06-01T20:53:20.963042+00:00
# this occurs x5, x1 for each run

scheduler_1  | [2020-06-01 20:43:28,856] INFO - prefect-server.Scheduler | Scheduled 5 flow runs.

scheduler_1  | [2020-06-01 20:43:28,972] DEBUG - prefect-server.api.schedules | Schedule <schedule_id> was not ready for new scheduling.

scheduler_1  | [2020-06-01 20:43:28,972] INFO - prefect-server.Scheduler | Scheduled 0 flow runs.

scheduler_1  | [2020-06-01 20:43:29,073] DEBUG - prefect-server.Scheduler | Sleeping for 300.0 seconds...