Matt Drago

    Matt Drago

    1 year ago
    Hey folks, I'm looking for some guidance on using the
    GCRResult
    . I have Prefect running in a kubernetes cluster, using the
    KubernetesRun
    with
    Docker
    storage. I've set the created the
    GCRResult
    using:
    result = GCSResult(
        bucket='REDACTED',
        location='flow-results/{flow_name}/{date:%Y/%m/%d}/{task_name}_{task_run_id}.prefect'
    )
    And assigned the result to the flow with:
    with Flow("Redacted", schedule=schedule, result=result) as flow:
    Weird thing is that the bucket name is being used to store the Result files, but not using the location template that I have provided. One thing that I noticed was that for one task (a function with the
    @task
    decorator), I passed in an argument with the name
    date
    and that Task did have it's Results stored in the location according to the template.
    Chris White

    Chris White

    1 year ago
    Hi @Matt Drago - are you putting different
    results
    on your tasks, or are you specifying a
    target
    on your tasks? If not, we might need to transition to a GitHub issue with a reproducible example to debug further
    Matt Drago

    Matt Drago

    1 year ago
    I'm only setting a
    result
    on the flow (note
    result
    singular not plural) and I'm not setting a
    target
    on any of the Tasks.
    Chris White

    Chris White

    1 year ago
    Hm yea then this sounds like a possible issue; for your own knowledge, this is the line where formatting occurs: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/engine/task_runner.py#L880-L893 As you can see there, the formatting kwargs are taken first from the flow’s parameters, then context, then finally from the task’s own keyword arguments. If you can create a minimal reproducible example and open an issue on GitHub we can get to the bottom of it! Feel free to tag me
    @cicdw
    if you do open one
    Matt Drago

    Matt Drago

    1 year ago
    Sweet, thanks for the pointer to the code as well. I'll try and get an example working and then raise an issue.
    Hey @Chris White, I spent some more time looking at the code that does the formatting and what I was getting. I misinterpreted my results. Prefect is working as intended. The thing that threw me was when I was setting the argument on a task to have the same name as what was in the context. The location template that I provide is being used and the results are going into the right place. On https://docs.prefect.io/core/concepts/results.html#templating-result-locations where it says:
    alternatively, you can provide a location template format string which will be templated with values in Prefect context at runtime.
    Do you think it would be worth changing this to mention that the template will use the values from the run's parameters, the context and then that tasks arguments and explain the preference for which set has precedence?
    Thanks for helping me out on this one.
    Chris White

    Chris White

    1 year ago
    Yea, I definitely think we should update the wording there; and anytime! Glad you were able to identify the issue — I’ll archive this thread to GitHub for others’ reference
    @Marvin archive “Result location template not being templated as I expect”
    Marvin

    Marvin

    1 year ago