Hi guys, firstly very cool tool! We're looking to improve our current ETL/data processing/business process flow by using Prefect as an orchestration tool. Our heavy lifting is done by R code, which is also exposed via a Plumber API on Kubernetes and some legacy processing engine(also converted to run in a container). Do you guys think Prefect will be a good fit? Basically I'm envisioning to stitch everything together with Python/Prefect, but what tasks should I use? Python REST calls, or should I rather do Shell Tasks and call the R code directly? (Apologies in advance if this type of question was already asked on the group)

Welcome @Hendrik Petzer! Of course Prefect will be a good fit 😉 I don’t think the community has agreed on the “one true way” to call R yet, but the most common thing we see is using a Shell task to run the R scripts, as you described. I think this probably provides the easiest adoption path while maintaining flexibility with your existing code. However, we designed Prefect to fit your workflow, not the other way around, so we encourage you to build tasks that mirror how you think about each unit of work — call R scripts when appropriate, mix in REST calls when you have a tightly scoped API, etc. Our team is here to provide any guidance as you build.

Great thanks for the feedback. I'll play around a bit to see what works for us.

👌