Hi! I would like to attach some result metadata as key-value pairs to each flow run, so that I could capture some basic metadata like “# of rows copied” or “# of rows in table” and stuff like that. Preferably I would not use results mechanism as having an external storage location feels like an overkill. Is there a way to achieve this?
Atte Keinänen
10/02/2023, 7:27 AM
Are artefacts the way to go at the moment?
Atte Keinänen
10/02/2023, 7:51 AM
Seems like they are. Will learn next how to read those artifacts via API, as we might have downstream use for some artifacts (e.g. persisting them to data warehouse, etc)
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.