JC Garcia
11/30/2020, 3:30 PMdbt_task = DbtShellTask(
...
)
with Flow("dbt-flow") as flow:
dbt_command_string = Parameter('dbt_command_string', default = 'dbt run', required = True)
dbt_commands = dbt_command_string.split(',')
for command in dbt_commands:
dbt_task(
command=dbt_command,
...
)
Zanie
Parameter
type then, not a string. You’d want to map over the parameter instead.Zanie
JC Garcia
11/30/2020, 4:52 PMZanie
matta
11/30/2020, 7:12 PMZanie
Zanie
DbtShellTask
functionality but want to track the running of the commands with a single Prefect task, you can pass the parameter to a task that does split and loop as you have written but call dbt_task.run(...)
within your task. The benefit of the (more complex) LOOP
pattern is that each of the subtasks would be tracked by Prefect but if that’s not important to you, calling .run
directly may be useful.matta
11/30/2020, 7:45 PMJC Garcia
12/01/2020, 1:57 AM