Hi there! I’ve written a task (it happens to be a function task, but it doesn’t have to be - could be a class that inherits from Task) that runs a nomad job. It polls for state and throws any exceptions up the stack. If the job fails, prefect is made aware of it and it works fine. I’m trying to understand how to go the other direction. If I cancel a flow run - how can I tell my Nomad job to stop? Is the thing to use here a
state_handler
? It’s sort of surprising to me that, for example, the kubernetes
This is a good question, and I think we don’t currently have support for it. I think it falls under thsi . It’s on the roadmap though
k
Kevin Weiler
09/24/2021, 2:23 PM
so I think this is a bit different - in this case K8s or Dask are being used as executors
Kevin Weiler
09/24/2021, 2:24 PM
I haven’t written a Nomad executor - my flow actually just runs inside a docker container and then calls out to the Nomad REST API
k
Kevin Kho
09/24/2021, 2:28 PM
Ah ok I see what you mean. So if the Flow is cancelled, the current shutdown is aggressive and that’s the spirit of the issue above. If the Flow fails though, you are right that a state handler will be able to help you, but for the
RunNamespaceJob
, I have seen people have difficulty with passing the
pod
details or
job
details out to that they can shut it down in the state handler.
I think if it fails, you are right that the state handler is a good approach if you can get the necessary info in there.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.