What's the best way to incrementally load tables (e.x. a loop) but then also ensure that if a connection is lost, the task retry will re-establish the connection AND not create duplicates on previously loaded rows?

You can persist a watermark in the KV store to keep track of the last processed date then update it after an operation

Thank you!