Kyle Foreman (Convoy)
08/29/2019, 5:16 PMChris White
task_run_count
which is found in context. So:
if prefect.context.get("task_run_count") == 1:
raise ValueError("fake error")
^^ will only fail on the first run; obviously you can tweak this to fail more than one time and eventually succeed, etc.Kyle Foreman (Convoy)
08/29/2019, 5:20 PMdef full_refresh_on_retry(dbt_task, old_state, new_state):
if isinstance(new_state, state.Retrying):
dbt_task.__recompile_node(full_refresh=True)
return new_state
Chris White
Kyle Foreman (Convoy)
08/29/2019, 5:37 PM[2019-08-29 17:34:49,545] INFO - prefect.TaskRunner | Unexpected error: ValueError('Fake error to test retries',)
[2019-08-29 17:34:51,865] ERROR - prefect.TaskRunner | Exception raised while calling state handlers: AttributeError("'DbtTask' object has no attribute '__recompile_node'",)
Chris White
Kyle Foreman (Convoy)
08/29/2019, 5:48 PMChris White
from IPython import embed; embed()
and make sure dbt_task
is what we think it isKyle Foreman (Convoy)
08/29/2019, 5:53 PM(Pdb++) dir(dbt_task)
['_DbtTask__build_runner', '_DbtTask__recompile_node', '__add__', '__and__', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__mifflin__', '__mod__', '__module__', '__mul__', '__ne__', '__new__', '__or__', '__pow__', '__radd__', '__rand__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rsub__', '__rtruediv__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__weakref__', '_compiled_node', '_dbt_flow', '_node_type', '_runner', 'auto_generated', 'bind', 'cache_for', 'cache_key', 'cache_validator', 'checkpoint', 'copy', 'inputs', 'is_equal', 'is_not_equal', 'logger', 'map', 'max_retries', 'name', 'not_', 'or_', 'outputs', 'result_handler', 'retry_delay', 'run', 'serialize', 'set_dependencies', 'set_downstream', 'set_upstream', 'skip_on_upstream_skip', 'slug', 'state_handlers', 'tags', 'timeout', 'trigger']
Chris White
Kyle Foreman (Convoy)
08/29/2019, 7:42 PM'_DbtTask__recompile_node'
is the methodChris White
Kyle Foreman (Convoy)
08/29/2019, 7:42 PMChris White
In [3]: class A:
...: __x = 5
...:
In [4]: a = A()
In [5]: a.__x
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-5-99a8eeb8b502> in <module>
----> 1 a.__x
AttributeError: 'A' object has no attribute '__x'
Kyle Foreman (Convoy)
08/29/2019, 7:44 PMChris White
__X
and __X_
Kyle Foreman (Convoy)
08/29/2019, 7:47 PMChris White
Jeremiah
KJ
08/30/2019, 12:11 AMKyle Foreman (Convoy)
08/30/2019, 1:30 AM