Hi everyone :wave:, Can someone assist with this q...
# prefect-community
Hi everyone 👋, Can someone assist with this question please? I simply try to backfill data by passing a datetime parameter https://stackoverflow.com/questions/64029629/is-there-a-way-to-backfill-historical-data-once-for-a-new-flow-in-prefect @Chris White gave an answer, but I can’t get it to work at all.
@Newskooler in the example with Context are you passing in
and accessing it as
in the task? That looks to be a typo in the answer
Hi @josh you are right, which is the correct way to write it in this case?
I wrote
current_time = prefect.context.get("intended_backfill_start_time") or prefect.context.get("scheduled_start_time")
and then have this unchanged like so
flow.run(context={"intended_backfill_start_time": "2020-09-01"}) # will use old timestamp
(it works).
Now what @Chris White is suggesting is to loop over the
until today, right?
Doesn’t matter which you write. Context here is just accessing an item from a dictionary and therefore the names need to match. Yeah next step would be to loop over a period of time and create the runs
Okay, so why do i not see all past such flow runs in the UI - if I run it and then register it (with a schedule) 🤔
(My thinking is too much Airflow like I guess, since I see past backfilled runs there, hence my confusion)
You should be seeing runs in the UI for each one you create when looping through time. The runs won’t be technically scheduled for a time in the past though because when you create the run you’re saying “Run this right now but perform some computation using data from the intended_backfill_start_time”
Oh also to note: if you’re calling
you won’t see them in the UI, that’s for local in process runs. You’re going to want to use the client
in order to create the flow run in the backend
How should I run in order to see them in the UI and to run them 🤔 : )
I recommend running through this deployment tutorial which will explain some of these concepts better: https://docs.prefect.io/orchestration/tutorial/first.html#write-flow
👍 1
Thanks. I will read now and may ask again later, if that’s okay @josh: )
👍 1