Hi there! I’ve written a task (it happens to be a ...
# ask-community
k
Hi there! I’ve written a task (it happens to be a function task, but it doesn’t have to be - could be a class that inherits from Task) that runs a nomad job. It polls for state and throws any exceptions up the stack. If the job fails, prefect is made aware of it and it works fine. I’m trying to understand how to go the other direction. If I cancel a flow run - how can I tell my Nomad job to stop? Is the thing to use here a
state_handler
? It’s sort of surprising to me that, for example, the kubernetes
RunNamespaceJob
doesn’t seem to include any logic around this (https://github.com/PrefectHQ/prefect/blob/master/src/prefect/tasks/kubernetes/job.py#L580).
k
This is a good question, and I think we don’t currently have support for it. I think it falls under thsi . It’s on the roadmap though
k
so I think this is a bit different - in this case K8s or Dask are being used as executors
I haven’t written a Nomad executor - my flow actually just runs inside a docker container and then calls out to the Nomad REST API
k
Ah ok I see what you mean. So if the Flow is cancelled, the current shutdown is aggressive and that’s the spirit of the issue above. If the Flow fails though, you are right that a state handler will be able to help you, but for the
RunNamespaceJob
, I have seen people have difficulty with passing the
pod
details or
job
details out to that they can shut it down in the state handler. I think if it fails, you are right that the state handler is a good approach if you can get the necessary info in there.
k
Ok cool - I’m going to read a bit more on the
state_handler
bit then. Seems to me that the
Task
metaclass should have a method stub like:
Copy code
def do_this_on_cancel():
    ...

def do_this_on_fail():
    ...
But I’m no developer…
k
The state handler supports that with:
Copy code
def mystatehandler(task,old_state,new_state):
    if new_state.is_failed():
       # logic
Btw have you seen Resource Managers as well? Docs . I still think state handler is the way to go but might be relevant