Is it safe to add a table into orion.db? or a column into flow_run ?
We would need to "log" additional info about flow results, but using "normal" logging may be not optimal, as log table may be big and we would need to do searching in text column...
08/12/2022, 10:39 AM
I wouldn't do it, it will make it super hard to do DB migrations once you upgrade to new versions
you can add all the log information using Prefect logger, why do you think adding a separate table for that would be beneficial?
there is already a table for that btw
08/12/2022, 10:45 AM
I have worries that using existing "log" table will be slow
Because it is a big table
and to store information that I can find later I have to log it in some specific way:
for example "my_unique_prefix_that_identifies_interesting_log: real message"
And searching these rows would require text search and that can be slow ?
08/12/2022, 11:58 AM
Up to you, I wouldn't do it
You can always store extra data/logs in other data stores
08/12/2022, 12:52 PM
yep, that's why I'm asking. If it is not recomended, second database will be an option