I have to confess right out of the gate, I posted a similar question to the Prefect Discourse, but it seems that the Slack channel is much more active, so after waiting about 2 weeks, I am going to try here.
I guess my basic question is, is it possible to programmatically specify the Job Templates in a workpool, either via command line or using Python? I want to specify
nodeSelector
,
tolerations
but there are many uses cases where one may want to change the K8s job manifest. I seem to be able to get the desired results changing it in the UI, but that requires logging in and changing it - I'm trying to automate as much as possible.
I see that the documentation says, "For advanced use cases, users can create work pools with fully customizable job templates. This customization is available when creating or editing a work pool on the 'Advanced' tab within the UI." Is that the only way?
Thanks,
Matt
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.