Thread
#prefect-community
    o

    Ogaday

    2 years ago
    I'm having some trouble with prefect Context. According to the docs, I can use context for time sensitive tasks and backfills (https://docs.prefect.io/core/faq.html#does-prefect-support-backfills). However, I've had a number of problems with this approach. I can't supply a datetime to the context of a flow which is run on a schedule, as that datetime is never recalculated after the initialisation of the flow run. And I can't seem to overwrite the default time values in the prefect context as described here: https://docs.prefect.io/core/concepts/execution.html#context Finally, a weird thing is that the date values (such as prefect.context.today) seems to be a strings, while the datetime values (such as prefect.context.date) are datetime objects. Am I doing anything wrong? I'm on Prefect 0.9.8 In the meantime, I'm going to look at using parameters instead of context.
    Chris White

    Chris White

    2 years ago
    Hi @Ogaday - any backfill run will need to be run off schedule. If you are using Core alone, you can call
    flow.run(run_on_schedule=False)
    with whatever context information you want to pass. In general, mixing ad-hoc runs with scheduled runs is a better experience when you use the Prefect API (either the newly open sourced prefect server, which requires 0.10.0+, or Prefect Cloud). And you are correct - some values of context are datetimes and others are strings; this is to support certain values and types that Airflow users would expect. The full list can be found here: https://docs.prefect.io/api/latest/utilities/context.html
    o

    Ogaday

    2 years ago
    Thanks @Chris White - the main problem I'm having is that I can't overwrite the default context, so I can't see how to run a backfill. In the example above, I'd expect the pipeline to print
    foo
    and
    hello
    but it prints a timestamp and the date.
    Chris White

    Chris White

    2 years ago
    Oh I see! Yes in this case you’ll want to pass context as follows:
    flow.run(context=dict(date='foo', my_date=datetime.utcnow(), today='hello'))
    Let me know if that does the trick
    o

    Ogaday

    2 years ago
    Thanks @Chris White that works perfectly. In that case it looks like the right thing to do is to use the default context items and overwrite them when I want to run the flow off schedule, like the docs/you suggest. I'll also look into the open source Prefect Orchestrator - haven't had a chance since it was announced last week, but it's super exciting news and I'm keen to get to grips with it!
    Chris White

    Chris White

    2 years ago
    Awesome! Let us know if you have any more questions, always happy to help
    @Marvin archive “How to override Prefect context values for a flow run”
    Marvin

    Marvin

    2 years ago