m

    Martim Lobao

    8 months ago
    we’ve got a large DAG with the same task repeated in several places using different inputs (do X for A, do X for B, etc), and it can be hard to quickly understand which task failed or even just understand the structure of the DAG itself. I tried using both the
    task_run_name
    and
    name
    args in the
    task
    decorator to provide more context, but neither works the way I’d like it to. as a MWE, here’s a sketch of what I’d like to happen:
    with Flow() as flow:
        task_a(entity="foo")  # shows up as "foo_task" in the DAG
        task_a(entity="bar")  # shows up as "bar_task" in the DAG
    the issue is that
    name
    only takes in static strings (so
    name="{entity}"
    doesn’t work) and
    task_run_name
    only sets the task run name, meaning it will never show up in the schematic outside of flow runs (even in flow runs, the name is only shown when clicking on each individual task card, making it hard to see an overall picture). is there any way to achieve what i’d like to do?
    Anna Geller

    Anna Geller

    8 months ago
    Generally speaking you have two opitions:1) Setting this as task_run_name as described in this page
    @task(task_run_name="{entity}")
    def task_a(entity):
    2) Use task_args
    with Flow() as flow:
        task_a(entity="foo", task_args={"name": "foo_task"})  # shows up as "foo_task" in the DAG
    btw, you can use both simultaneously, since task_name and task_run_name are two different things
    m

    Martim Lobao

    8 months ago
    ah, perfect, task_args seems to be the option i was looking for 🙂
    thanks Anna!