https://prefect.io logo
Title
m

Marcin Grzybowski

08/12/2022, 9:07 AM
HI all, Is it safe to add a table into orion.db? or a column into flow_run ? We would need to "log" additional info about flow results, but using "normal" logging may be not optimal, as log table may be big and we would need to do searching in text column...
1
a

Anna Geller

08/12/2022, 10:39 AM
I wouldn't do it, it will make it super hard to do DB migrations once you upgrade to new versions you can add all the log information using Prefect logger, why do you think adding a separate table for that would be beneficial?
there is already a table for that btw
m

Marcin Grzybowski

08/12/2022, 10:45 AM
I have worries that using existing "log" table will be slow Because it is a big table and to store information that I can find later I have to log it in some specific way: for example "my_unique_prefix_that_identifies_interesting_log: real message" And searching these rows would require text search and that can be slow ?
a

Anna Geller

08/12/2022, 11:58 AM
Up to you, I wouldn't do it You can always store extra data/logs in other data stores
m

Marcin Grzybowski

08/12/2022, 12:52 PM
yep, that's why I'm asking. If it is not recomended, second database will be an option
💯 1
thank you
🙌 1