This is probably a result of being unfamiliar with Prefect's verbology... but I'm having a hard time figuring out how to share state between flow runs. I am building a flow that will need store a "cursor"-like value between runs
r
redsquare
04/05/2023, 5:41 AM
You could use a json block to store a 'checkpoint' that you read in on the next run, or persist it to a db
m
Matt Fysh
04/05/2023, 12:28 PM
ok I think I got the hang of it, I had to create the json block in the UI before running the job, and then to update the state I add ovewrite=True
now just trying to dig through the sqlite prefect.db file, but I can't seem to find where the value is stored, have done select * on most tables
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.