Deceivious08/02/2023, 9:32 AM
table. Question and details in thread. TLDR: Why are there so many
Local File System
table, even when I am not using Local File system directly on my code? Is it safe to just delete them?
table. I did a quick analysis.
Results are as follows:
select bt.name,count(1) from block_document bd , block_type bt where bt.id=bd.block_type_id and bd.updated >= now() - interval '1 week' group by bt.name;
NOTE: This is data for only 1 week. I decrypted the
"name","count" Kubernetes Job,19 Azure,1 Date Time,11 Local File System,17072
with the Fernet key. The decrypted values are the same and refer to the PREFECT_DIRECTORY base path. This is pain ful as the table is estimated to be 50+ GB for us and most of the values are just encrypted duplicates of the same value.
This created a single record in block document. I took the dump sql of the entire database in text format, searched for the ID of the block document record and there are no references to that ID in any other tables.
import prefect from prefect import flow, task @task def ta(): prefect.get_run_logger().info("THis is from task") @flow def sub_flow(): prefect.get_run_logger().info("This is sub flow") ta() @flow def main_flow(): prefect.get_run_logger().info("This is main flow") sub_flow() ta() if __name__ == "__main__": main_flow()
*delete instead of select
select bd.* from block_document bd where bd.updated < now()-interval '1 week' and is_anonymous = True and exists ( select 1 from block_type bt where bt.id=bd.block_type_id and bt.name='Local File System' )
Jake Kaplan08/02/2023, 1:40 PM
Deceivious08/02/2023, 1:41 PM