Florian Kühnlenz
11/08/2022, 2:19 PMGordon Silvera
11/08/2022, 2:51 PMAttributeError: module 'prefect.tasks' has no attribute 'dbt'
. I've tried to run this locally and within the jupyter-scipy Docker image. Any thoughts why Prefect cannot find the module?
# dbt_shell_task.py
import sys
import prefect
from prefect import task, flow, get_run_logger
@flow
def dbt_flow(cmd='dbt run'):
# Execute specified command
task = prefect.tasks.dbt.dbt.DbtShellTask(
command=cmd,
profile_name='default',
environment='Development',
dbt_kwargs={'type': 'bigquery'},
overwrite_profiles=False,
profiles_dir='/home/jovyan/.dbt/profiles.yml'
)
logger = get_run_logger()
<http://logger.info|logger.info>("Command Run: %s!", name)
return task
if __name__ == "__main__":
cmd = sys.argv[1]
dbt_flow(cmd)
# packages
prefect==2.6.6
prefect-dbt==0.2.4
prefect-shell==0.1.3
dbt-bigquery==1.3.0
dbt-core==1.3.0
dbt-extractor==0.4.1
Jared Robbins
11/08/2022, 3:45 PMAxel Sundberg
11/08/2022, 3:55 PM@task
annotation. I have defined two tasks in a flow like this:
@task(name="Backup product catalogue", log_stdout=True)
def backup_task():
backup_product_catalogue_to_gcs()
@task(name="Update Product Catalogue", log_stdout=True, trigger=triggers.all_finished)
def upsert_pack_products_task():
upsert_pack_products()
with Flow(
name="Product catalogue",
run_config=get_run_config(),
storage=get_storage("product_catalogue/flow.py"),
) as product_catalogue_flow:
backup_task()
upsert_pack_products_task(upstream_tasks=[backup_task])
I renamed the function name upsert_pack_products_task
and re-deployed. Now I can see two Backup product catalogue
in my flow on Prefect and they both run (see image) when the flow is triggered. How can I remove that extra task? And how can I debug what is going on?Sunjay
11/08/2022, 4:13 PMJames Brady
11/08/2022, 5:19 PMCrash detected! Execution was interrupted by an unexpected exception.
(full logs in thread). I'm trying to figure out the best way to debug this further.
In general – and to help diagnose this particular case – I'd like to shuttle logs to our DataDog instance: I found this discussion on the topic, but is there something more recent / more complete for Prefect v2?
Aside from getting logs out, any other ideas for diagnosing this terse error?Madison Schott
11/08/2022, 5:23 PMsync = await fivetran_sync_flow(
^
SyntaxError: 'await' outside async function
Blake Stefansen
11/08/2022, 7:27 PMOrion logging error
with 2.6.6 for our k8 jobs
Will post logs in the threadDavid Beck
11/08/2022, 8:08 PMPaco Ibañez
11/08/2022, 10:43 PMRohith
11/09/2022, 8:40 AMPierre Monico
11/09/2022, 9:41 AMThomas Pedersen
11/09/2022, 10:05 AMDan Wise
11/09/2022, 11:48 AMToshali Narula
11/09/2022, 2:20 PMRevanth
11/09/2022, 2:33 PMVadym Dytyniak
11/09/2022, 3:09 PMKhuyen Tran
11/09/2022, 4:59 PMAlexandru Anghel
11/09/2022, 5:33 PMfrom tasks.date import get_start_date
@task(name='Generate start date', retries=2, retry_delay_seconds=10)(get_start_date)
It fails with `RuntimeError: Tasks cannot be run outside of a flow. To call the underlying task function outside of a flow use task.fn()
.`
Thanks!Kingsley
11/09/2022, 7:22 PMTim Enders
11/09/2022, 7:40 PMMadison Schott
11/09/2022, 10:07 PMScott McCallen
11/09/2022, 10:25 PMTim Ricablanca
11/09/2022, 11:55 PMRunning
states but not for Failed
— details in thread:Kishan
11/10/2022, 2:44 AMKishan
11/10/2022, 2:47 AMmerlin
11/10/2022, 4:46 AMValueError
to trigger a failure, and thats how to manage the state of subsequent tasks or flows.
Here's the thing: I don't want to see a stack trace unless it is some new failure that I need to troubleshoot. For the expected exceptions I want to post an error the log and proceed with my flow according to logic I design.
I'm having trouble returning a manual state of Cancelled, concrete example and logs posted in thread. Can anyone guide me toward causing the entire flow to be cancelled without raising an exception?Rohith
11/10/2022, 9:56 AMredsquare
11/10/2022, 10:01 AMDan Wise
11/10/2022, 11:18 AMDan Wise
11/10/2022, 11:18 AMChristopher Boyd
11/10/2022, 3:03 PMKhuyen Tran
11/10/2022, 4:04 PMDan Wise
11/10/2022, 4:34 PMKhuyen Tran
11/10/2022, 4:49 PMWhen I deploy then run with an agent the decorator is not triggeredWhich version of prefect-alert are you in?
We use messages posted via an app using the slack client so was wondering it a notification type will be created for firing messages to Slack that wayYou mean creating notification using something that is not a notification block?
In the alert_on_failure decorator, you define the block_type to be an object instance of a AppriseNotificationBlock but I believe you are expecting a class instanceCould you be more specific about that you mean by object instance and class instance?
Dan Wise
11/10/2022, 4:55 PMdef alert_on_failure(block_type: AppriseNotificationBlock, block_name: str):
Khuyen Tran
11/10/2022, 7:41 PM