Hello,
I've noticed some Prefect API field names keeps changing every week, and this is breaking our pipeline that ingests metadata. Is there any way of getting notified when a field name changes, or some channel where those changes are being communicated in advance?
Your pipeline that ingests metadata about our API or it’s just breaking your flow?
j
Joao Moniz
02/04/2023, 4:55 PM
Yes, sorry. We have one pipeline that reaches the Prefect Cloud API to get our flow metadata (some endpoints like flow runs, task runs, deployments). We have some great expectations set to alert us if the schema is changing, and it has been triggered at least once a week for the past weeks, which means that some API fields are constantly changing names or having fields added/deleted. Just was wondering if there was a way of being alerted beforehand about those changes
z
Zanie
02/04/2023, 4:58 PM
Ah I see. It sounds like you're being alerted for changes of experimental fields. Perhaps we can label those in the schema and you can ignore them?
👍 1
Zanie
02/04/2023, 4:59 PM
We don't have a way to notify you beforehand right now.
j
Joao Moniz
02/06/2023, 8:23 AM
Ah that would be great, thanks 🙂
z
Zanie
02/06/2023, 3:56 PM
Would you mind opening a ticket on GitHub with a bit of details about your use-case?
👍 1
Zanie
02/06/2023, 3:56 PM
That’ll help our product team understand it, I don’t think many people are doing this 😄
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.