JC Garcia
11/30/2020, 3:30 PMdbt_task = DbtShellTask(
...
)
with Flow("dbt-flow") as flow:
dbt_command_string = Parameter('dbt_command_string', default = 'dbt run', required = True)
dbt_commands = dbt_command_string.split(',')
for command in dbt_commands:
dbt_task(
command=dbt_command,
...
)
Zanie
11/30/2020, 4:49 PMParameter
type then, not a string. You’d want to map over the parameter instead.JC Garcia
11/30/2020, 4:52 PMZanie
11/30/2020, 6:45 PMmatta
11/30/2020, 7:12 PMZanie
11/30/2020, 7:19 PMDbtShellTask
functionality but want to track the running of the commands with a single Prefect task, you can pass the parameter to a task that does split and loop as you have written but call dbt_task.run(...)
within your task. The benefit of the (more complex) LOOP
pattern is that each of the subtasks would be tracked by Prefect but if that’s not important to you, calling .run
directly may be useful.matta
11/30/2020, 7:45 PMJC Garcia
12/01/2020, 1:57 AM