Hi guys, our company has multiple nomad clusters on-prem and I'm looking into the possibility of setting up a data platform on one of our clusters.
Prefect is one of the candidate tools we're looking at as part of our data platform. My plan would be to deploy Prefect Server into our nomad cluster, and then configure a Custom Worker to setup nomad as an execution environment so we can use our nomad servers as worker nodes for Prefect.
I was wondering if the community has any tips/gotchas on implementing custom workers generally, and also nomad specifically. Appreciate your input! Thanks
n
Nate
06/08/2024, 12:45 AM
hi @Brian T - I just made one small fix but besides that, no real gotchas I can think of!
these could be good reference:
• existing workers
◦
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.