Madison Schott
02/08/2022, 8:19 PMre_data notify slack \
--start-date 2021-01-01 \
--end-date 2021-01-31 \
--webhook-url <https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX> \
--subtitle="[Optional] Markdown text to be added as a subtitle in the slack message generated"
Madison Schott
02/08/2022, 8:22 PMfrom prefect.tasks.shell import ShellTask
task = ShellTask(helper_script="cd ~")
re_data_slack = task(command='re_data notify slack')
Madison Schott
02/08/2022, 8:23 PMMadison Schott
02/08/2022, 8:24 PMRUN pip install re_data
in the Dockerfile I'm using?Kevin Kho
Kevin Kho
pip install re-data
if it’s this . you might be using the underscoreMadison Schott
02/09/2022, 6:05 PMMadison Schott
02/09/2022, 6:05 PMfrom prefect.tasks.shell import ShellTask
shell_task = ShellTask(helper_script="cd ~")
re_data_slack_alert = shell_task(command='re_data notify slack \
--start-date 2022-01-01 \
--end-date 2022-07-31 \
--webhook-url xxx')
Kevin Kho
log_stderr=True
and return_all=True
and stream_output=True
to the shell task so we can try to get more logs?>Kevin Kho
Madison Schott
02/09/2022, 8:09 PMKevin Kho
Madison Schott
02/10/2022, 10:31 PMMadison Schott
02/10/2022, 10:31 PMshell_task = ShellTask(helper_script="cd /dbt_snowflake/", log_stderr=True, return_all=True, stream_output=True)
Madison Schott
02/10/2022, 10:32 PMKevin Kho
Madison Schott
02/11/2022, 4:53 PMMadison Schott
02/11/2022, 4:53 PMpackages.yml
Madison Schott
02/11/2022, 4:54 PMdbt deps
?Kevin Kho
Madison Schott
02/11/2022, 5:29 PMMadison Schott
02/11/2022, 5:31 PMStep 6/14 : RUN pip install re_data
---> Running in 8a00459332d9
Collecting re_data
Downloading re_data-0.6.0-py3-none-any.whl (543 kB)
Madison Schott
02/11/2022, 5:31 PMKevin Kho
Madison Schott
02/11/2022, 5:39 PMRUN_CONFIG = ECSRun(run_task_kwargs={'cluster': 'prefect-prod'},
env={"PREFECT__LOGGING__LEVEL": "DEBUG"},
execution_role_arn='arn:aws:iam::xx',
labels=['ecs-agent', 'prod', 'winc'])
Kevin Kho
Madison Schott
02/11/2022, 6:30 PMKevin Kho