Hey everyone! I am new to prefect and I am looking to implement a POC that mimics an implementation we have going in Airflow.
All of our DAGs in Airflow use KubernetesPodOperators + a few hooks for each individual task.
What's the best way to emulate this in Prefect?
I have a self-hosted Prefect instance going on top of Kubernetes, and I have a Kubernetes work pool going. From what I have been reading/gathering, we can only deploy flows as kubernetes pods, and each individual task gets executed within that pod.
Is it possible to run individual tasks as k8s pods?
I think trying to emulate this workflow in Prefect is a little weird since we would be spinning up a pod to run a flow that also spins up other pods... all via the work pool pod. I am wondering if anyone has had a similar experience?