Hey all, I’m Tom - working on automating PPC optimization.
We’ve been struggling for years to figure out a good solution for our ingestion pipelines. We have a fairly complicated DAG that needs to be run fully, without errors, before we can actually run the automations.
Discovered Prefect recently and I’ve been trying to pick my jaw up off the floor since then. It abstracts away so much of the complicated stuff we had needed to put into our pipeline.
Currently working on building out a proof of concept. Eager to see where it goes!