Hey all! Wanted to share my org's story on migrating from Civis (an analyst-friendly but limited orchestration tool) to Prefect. Here's my blog post about it. I cover the limitations of Civis vis a vis data engineering best practices and how Prefect improves our workflows. I cover what it looks like to migrate a script from Civis to Prefect. I get into details about how we set up our execution layer in AWS ECS based on the dataflowops tutorial with a few modifications. I include a sample repository with working template scripts. I also demo a modification to Prefect's concurrency implementation to support rate limiting API calls. I always appreciate seeing demos of how folks go about their implementation and hope this is helpful if you're looking into how to set up Prefect in AWS ECS or move off of a limited legacy orchestration stack!

Nice, thanks for sharing! @Will Raphaelson used to work on Civis and also migrated (himself) to Prefect, so I’m sure he’d be excited to hear about this!