Eric Mauser07/27/2021, 8:52 PM
locally it returns that both tasks were successful, however I can see that it didn't actually run the dbt package. Any thoughts here? The dbt package is just in the same folder as the
. I'm assuming this all gets packaged up and run on ECS.
Eric Mauser07/27/2021, 8:53 PM
import prefect from prefect.tasks.dbt import DbtShellTask from prefect.storage import Docker from prefect.run_configs import ECSRun from prefect import task, Flow, Parameter STORAGE = Docker(registry_url='<myrole>', image_name='dbt', image_tag='latest') RUN_CONFIG = ECSRun(execution_role_arn='<myrole>', image = 'dbt', labels=['dev']) with Flow('dbt', storage=STORAGE, run_config=RUN_CONFIG) as flow: task = DbtShellTask(profile_name='default', profiles_dir='')(command = 'dbt deps') task2 = DbtShellTask(profile_name='default', profiles_dir='')(command = 'dbt run') flow.register('<myproject>') flow.run()
and `return_all=True`as seen here so we can get the logs from the Shell Task? Also, I guess you can try using the container built and running those commands without Prefect to see if they work?
Eric Mauser07/27/2021, 9:34 PM
or do I need to build a custom image that combines prefect and dbt?