Hey Prefect team! I was wondering if there's documentation or a github repo that I can look at for integrating Singer taps with Prefect? What I would like to do is run Singer taps in a prefect flow. I did some research and I see Singer tap integrations with Airflow and Dagster so I assume Prefect has the capabilities too but I can't seem to figure it out 😅
k
Kevin Kho
03/23/2022, 8:48 PM
You are the second person to task. I’ve yet to see any content around it. I think more of our users use Airbyte for connections to sources
Kevin Kho
03/23/2022, 8:48 PM
Are you tied to Singer? or open to Airbyte?
upvote 1
e
Edward Chen
03/23/2022, 9:16 PM
We're still evaluating between Airbyte and Singer so that's why I've been using Singer. One of our hesitation with Airbyte is that it's still early on so I remember reading somewhere that it may not be production-ready.
a
Anna Geller
03/23/2022, 9:28 PM
the term production-ready really doesn't mean anything anymore these days 🙂 how would you assess whether a tool is production ready?
we do have users using Prefect's Airbyte integration in production
e
Edward Chen
03/23/2022, 9:51 PM
how would you assess whether a tool is production ready?
Hahaha you have a good point 😂
In the meantime while I try out how easy things are to do, is there any way to integrate Singer taps with Prefect?
EDIT: Ah I see there's something with ShellTask
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.