Yeah, that is what I was trying to remember, but I dont think it is what I want. Trying to work around BQ rate limits on writing some context (and other) metadata to the db
Wait, is there a doc/definition for the GraphQL API somewhere?
k
Kevin Kho
01/20/2022, 7:05 PM
Oh I don’t think there is room for that to store metadata in the Prefect database. The documentation would be in the
Interactive API
tab of the UI
How big is this metadata?
t
Tim Enders
01/21/2022, 2:26 PM
It is some pieces of the context and then some metrics from the run. Record counts, page status on APIs
k
Kevin Kho
01/21/2022, 2:37 PM
I think conceptually using a task to output a result would make the most sense. We don’t have any place in the database to insert information.
t
Tim Enders
01/21/2022, 2:41 PM
Yeah, I just need to write data form multiple tasks to one metadata table. And I don't want to stream the data in if I don't have to. Thinking a Flow Run ID named file will be unique enough for me to stash it all and write at the end of the Flow.