Thread
#prefect-community
    a

    ARun

    2 years ago
    hey guys, got a basic questions. Just found out about prefect, and started messing around with it today. I've been following along the docs. And I've tried using this locally, and using the dask executor. I think I'm missing something. I've made a simple job to move file from a database into a sftp. However, when I run the file, I get this message. If I ctrl-c out of it, it crashes the program. Do i have to do it in nohup, or is there a way i can execute the script in the background. So it just runs, and follows the schedule? If there is any documentation on this, that would be of great help. Thanks!
    [2019-12-05 18:38:18,226] INFO - prefect.Flow: sf -> sftp | Waiting for next scheduled run at 2019-12-05T21:00:00+00:00
    j

    josh

    2 years ago
    What is the schedule you’re attaching to your Flow?
    Ultimately the schedule needs to run in a process so you are able to spin up however you want! Whether it’s in a subprocess, docker container, etc…
    a

    ARun

    2 years ago
    The value is dynamically generated from some options
    That's what the code looks like
    j

    josh

    2 years ago
    So the reason your schedule sits there is because it’s waiting for the scheduled run at 21:00 and the reason why it ends when you quit the process is because the schedule is attached to the flow. When the process ends the Flow object that you created no longer exists. I recommend using something like supervisor if you want to run this in the background https://pypi.org/project/supervisor/
    a

    ARun

    2 years ago
    Ah, I see. Thanks
    Is there a way to manage all the subprocess via prefect?
    j

    josh

    2 years ago
    All of the subprocesses on your machine running schedules would be outside of prefect’s control
    a

    ARun

    2 years ago
    thanks