https://prefect.io logo
Title
g

Greg

09/23/2020, 3:19 PM
Hi ! I've updated server, now all tasks running for more than 15min are killed by the ZombieKiller ! How can I disable it ?
j

josh

09/23/2020, 3:42 PM
Hi @Greg the zombie killer should only be killing tasks who do not send a heartbeat after a set amount of time. Generally this means that your tasks are still in a running state but somehow are unable to send their heartbeat back to cloud. If you want to disable this feature you should be able to on your flow’s settings page by toggling off the heartbeat
Which version were you encountering this and which version does it not happen for you?
g

Greg

09/23/2020, 3:43 PM
I'm running Local Agent, and issue is coming from long running SQL queries
thanks for the tip to disable heartbeat by flow, however I have >100 flows and want to disable it by default
j

josh

09/23/2020, 3:47 PM
Could you open an issue on the Prefect repo with more information about the flow that is being zombie killed? Hard to say what’s causing it without more information
g

Greg

09/23/2020, 3:58 PM
that's basically all flows running a long SQL query (more than 15min)
typical task:
@task(tags=["extract"])
    def extract_mail_domain() -> pd.DataFrame:
        db = Db.Db("mydb")
        db.init_sqlalchemy(slave=True)

        sql_query = """
            SELECT
                id as mail_domain_id,
                domain as label,
                email_domain_group_id as mail_domain_group_id
            FROM email_domain ed
            LEFT JOIN email_domain_group_has_domain edghd ON (ed.id = edghd.email_domain_id)
        """

        dataframe = pd.read_sql_query(sql_query, db.get_engine(),)
        db.close()

        if len(dataframe) > 0:
            return dataframe

        raise prefect.engine.signals.SKIP()
(with pd = pandas)
j

josh

09/23/2020, 5:23 PM
Generally this is a resource related issue, take a look at this thread https://github.com/PrefectHQ/prefect/issues/1954 maybe it might lead to some help. Otherwise I would suggest opening an issue with your setup on how you are running your flows