https://prefect.io logo
Title
a

ARun

12/05/2019, 6:42 PM
hey guys, got a basic questions. Just found out about prefect, and started messing around with it today. I've been following along the docs. And I've tried using this locally, and using the dask executor. I think I'm missing something. I've made a simple job to move file from a database into a sftp. However, when I run the file, I get this message. If I ctrl-c out of it, it crashes the program. Do i have to do it in nohup, or is there a way i can execute the script in the background. So it just runs, and follows the schedule? If there is any documentation on this, that would be of great help. Thanks!
[2019-12-05 18:38:18,226] INFO - prefect.Flow: sf -> sftp | Waiting for next scheduled run at 2019-12-05T21:00:00+00:00
j

josh

12/05/2019, 6:46 PM
What is the schedule you’re attaching to your Flow?
Ultimately the schedule needs to run in a process so you are able to spin up however you want! Whether it’s in a subprocess, docker container, etc…
a

ARun

12/05/2019, 6:51 PM
The value is dynamically generated from some options
That's what the code looks like
j

josh

12/05/2019, 6:53 PM
So the reason your schedule sits there is because it’s waiting for the scheduled run at 21:00 and the reason why it ends when you quit the process is because the schedule is attached to the flow. When the process ends the Flow object that you created no longer exists. I recommend using something like supervisor if you want to run this in the background https://pypi.org/project/supervisor/
💯 1
a

ARun

12/05/2019, 6:55 PM
Ah, I see. Thanks
Is there a way to manage all the subprocess via prefect?
j

josh

12/05/2019, 6:59 PM
All of the subprocesses on your machine running schedules would be outside of prefect’s control
a

ARun

12/05/2019, 7:02 PM
thanks