n

    Nicholas Chammas

    1 year ago
    New Prefect user here. I’m trying to understand the difference between static and dynamic Task arguments, as mentioned in this user warning: https://github.com/PrefectHQ/prefect/blob/553544ca25e454140dda363c8f8ca648059a874c/src/prefect/core/task.py#L154-L156 The warning points to a part of the docs that mentions neither the word “static” nor the word “dynamic”, so I’m not sure what the authors really mean when they use these words.
    Perhaps there are some examples I could review that clarify this distinction between static and dynamic Task arguments?
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    That's a fair point, we should improve those docs & error message. I'll try to clarify here: • Static arguments mean "constants defined at flow definition time". Basically anything that isn't a task. The "(non-Task)" part of the error message was meant to indicate this. • Dynamic arguments mean anything that is a Task. Things we don't know the value of until the flow is actually executing. The "(Task)" part of the error message was meant to indicate this. In short - don't pass
    Task
    -type arguments to class-based
    Task
    constructors, pass them when calling the task instead.
    If you have any suggestions for how we could word this error message better, I'd love to hear them.
    Amanda Wee

    Amanda Wee

    1 year ago
    The definition seems to be given in the message itself: a "dynamic" argument is one that is a task; a "static" argument is one that is not a task. The idea is that if the argument is itself a task, it should be provided to this task when the task is run, not when it is initialised.
    n

    Nicholas Chammas

    1 year ago
    The distinction is starting to make sense as I get familiar with the functional API, but I wonder how this could be made clearer. For example, here are the docs for a Task I’m using: https://docs.prefect.io/api/latest/tasks/databricks.html#databricksrunnow There is no indication in the docs / API signature that
    databricks_conn_secret
    , for example, is not like the other input parameters to
    DatabricksRunNow
    .
    databricks_conn_secret
    is a
    PrefectSecret
    , which is a
    Task
    , which must thus be passed in after the
    DatabricksRunNow
    task is initialized with the other parameters. I wonder if there is a way the docs or even the API itself could be tweaked such that the separation between Task and non-Task parameters is somehow clearer or more obvious. Or does that not make sense? e.g. Because parameters can in theory always have either Task or non-Task inputs, depending on how we choose to obtain them. That would make sense. It still makes me wonder if there is a way this could have been done differently so that people don’t have to think about this distinction when initializing tasks. But I will probably understand the current API better as I continue to get familiar with it.