Andrea Nerla
02/04/2022, 3:08 PMFile "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\box\box.py", line 516, in __getattr__
value = object.__getattribute__(self, item)
AttributeError: 'Config' object has no attribute 'datefmt'
Traceback (most recent call last):
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\box\box.py", line 488, in __getitem__
return super().__getitem__(item)
KeyError: 'datefmt'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\box\box.py", line 514, in __getattr__
value = self.__getitem__(item, _ignore_default=True)
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\box\box.py", line 509, in __getitem__
raise BoxKeyError(str(err)) from _exception_cause(err)
box.exceptions.BoxKeyError: "'datefmt'"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\box\box.py", line 516, in __getattr__
value = object.__getattribute__(self, item)
AttributeError: 'Config' object has no attribute 'datefmt'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\Users\andrea.nerla\Desktop\sidal\sidal prefect\prefect test var risorsas.py", line 1, in <module>
import prefect
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\prefect\__init__.py", line 1, in <module>
import prefect.utilities
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\prefect\utilities\__init__.py", line 1, in <module>
import prefect.utilities.logging
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\prefect\utilities\logging.py", line 261, in <module>
context.logger = prefect_logger = configure_logging()
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\prefect\__init__.py", line 1, in <module>
import prefect.utilities
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\prefect\utilities\__init__.py", line 1, in <module>
import prefect.utilities.logging
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\prefect\utilities\logging.py", line 261, in <module>
context.logger = prefect_logger = configure_logging()
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\prefect\utilities\logging.py", line 258, in configure_logging
return _create_logger(name)
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\prefect\utilities\logging.py", line 230, in _create_logger
context.config.logging.format, context.config.logging.datefmt
File "C:\Users\andrea.nerla\AppData\Local\Programs\Python\Python39\lib\site-packages\box\box.py", line 530, in __getattr__
raise BoxKeyError(str(err)) from _exception_cause(err)
box.exceptions.BoxKeyError: "'Config' object has no attribute 'datefmt'"
Kevin Kho
flow.run()
or is this with an agent?Andrea Nerla
02/04/2022, 3:31 PMprefect agent local start
, which gives me this error. But before of it I was having the same error with a flow.run()
Kevin Kho
pip show python-box
in your terminal and show me the output?Andrea Nerla
02/04/2022, 3:35 PMKevin Kho
pip show prefect
?Andrea Nerla
02/04/2022, 3:37 PMKevin Kho
Andrea Nerla
02/04/2022, 3:57 PMKevin Kho
Kevin Otte
03/16/2022, 12:48 AMKevin Kho
flow.run()
no. For running with an agent against Prefect Cloud yes you have to be authenticatedKevin Otte
03/16/2022, 12:54 AM[context.secrets]
SLACK_WEBHOOK_URL = "<https://hooks.slack.com/services/xxx>"
to config.toml and using this sample code
from prefect import Flow, task
from prefect.utilities.notifications import slack_notifier
@task(name="1/x task", state_handlers=[slack_notifier])
def div(x):
return 1 / x
@task(name="Add 1 task", state_handlers=[slack_notifier])
def add(x):
return x + 1
with Flow('Add Divide Flow') as f:
res = div(x=add(-1))
final_state = f.run()
Kevin Kho
export PREFECT___CLOUD____USE_LOCAL_SECRETS=true_
[context.secrets]
SLACK_WEBHOOK_URL = "<https://hooks.slack.com/services/T015STTHK0A/B0342AESY6L/>
[cloud]
use_local_secrets=true
Kevin Otte
03/16/2022, 1:02 AMKevin Kho
Kevin Otte
03/16/2022, 1:04 AMKevin Kho
.prefect
folder and then editing it directly to put your Slack secret?[context.secrets]
SLACK_WEBHOOK_URL = "<https://hooks.slack.com/services/xxx/xxx/xxx>"
[cloud]
use_local_secrets = true
And then your script works with flow.run()
, we don’t need a backend yet to test thisKevin Otte
03/16/2022, 1:29 AMThe above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/runner.py", line 161, in handle_state_change
new_state = self.call_runner_target_handlers(old_state, new_state)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/task_runner.py", line 113, in call_runner_target_handlers
new_state = handler(self.task, old_state, new_state) or new_state
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/toolz/functoolz.py", line 306, in __call__
return self._partial(*args, **kwargs)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/notifications/notifications.py", line 316, in slack_notifier
form_data = slack_message_formatter(tracked_obj, new_state, backend_info)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/notifications/notifications.py", line 161, in slack_message_formatter
url = prefect.client.Client().get_cloud_url(
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/client/client.py", line 1008, in get_cloud_url
tenant_slug = self.get_default_tenant_slug()
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/client/client.py", line 1030, in get_default_tenant_slug
res = self.graphql({"query": {"tenant": {"id", "slug"}}})
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/client/client.py", line 452, in graphql
result = <http://self.post|self.post>(
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/client/client.py", line 407, in post
response = self._request(
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/client/client.py", line 649, in _request
raise AuthorizationError(
prefect.exceptions.AuthorizationError: Malformed response received from Cloud - please ensure that you are authenticated. See `prefect auth login --help`.
[2022-03-15 21:04:02-0400] INFO - prefect.TaskRunner | Task '1/x task': Finished task run for task with final state: 'Failed'
[2022-03-15 21:04:02-0400] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
(venv39) ➜ airflow-pipelines git:(minor_refactor) ✗ python test_slack.py
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/box/box.py", line 488, in __getitem__
return super().__getitem__(item)
KeyError: 'logging'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/box/box.py", line 514, in __getattr__
value = self.__getitem__(item, _ignore_default=True)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/box/box.py", line 509, in __getitem__
raise BoxKeyError(str(err)) from _exception_cause(err)
box.exceptions.BoxKeyError: "'logging'"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/box/box.py", line 516, in __getattr__
value = object.__getattribute__(self, item)
AttributeError: 'Config' object has no attribute 'logging'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/test_slack.py", line 1, in <module>
from prefect import Flow, task
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/__init__.py", line 1, in <module>
import prefect.utilities
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/__init__.py", line 1, in <module>
import prefect.utilities.logging
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/logging.py", line 280, in <module>
context.logger = prefect_logger = configure_logging()
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/logging.py", line 277, in configure_logging
return _create_logger(name)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/logging.py", line 249, in _create_logger
context.config.logging.format, context.config.logging.datefmt
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/box/box.py", line 530, in __getattr__
raise BoxKeyError(str(err)) from _exception_cause(err)
box.exceptions.BoxKeyError: "'Config' object has no attribute 'logging'"
Kevin Kho
slack_notifier
is coded to fetch the secret from Prefect Cloud. This is an old piece of code and the built-in SlackTask is preferred. You should be able to run this locally without Prefect Cloud:
from prefect import Flow, task
from prefect.tasks.notifications.slack_task import SlackTask
def mystatehandler(obj, old_state, new_state):
SlackTask().run(message="test message", webhook_secret="SLACK_WEBHOOK_URL")
return new_state
@task(name="1/x task", state_handlers=[mystatehandler])
def div(x):
return 1 / x
@task(name="Add 1 task", state_handlers=[mystatehandler])
def add(x):
return x + 1
with Flow('Add Divide Flow') as f:
res = div(x=add(-1))
final_state = f.run()
Kevin Otte
03/16/2022, 1:41 AM(venv39) ➜ airflow-pipelines git:(minor_refactor) ✗ python test_slack.py
[2022-03-15 21:40:08-0400] INFO - prefect.FlowRunner | Beginning Flow run for 'Add Divide Flow'
[2022-03-15 21:40:08-0400] INFO - prefect.TaskRunner | Task 'Add 1 task': Starting task run...
[2022-03-15 21:40:08-0400] ERROR - prefect.TaskRunner | Unexpected error while calling state handlers: HTTPError('403 Client Error: Forbidden for url: <https://hooks.slack.com/services/T015STTHK0A/B0342AESY6L/>')
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/runner.py", line 161, in handle_state_change
new_state = self.call_runner_target_handlers(old_state, new_state)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/task_runner.py", line 113, in call_runner_target_handlers
new_state = handler(self.task, old_state, new_state) or new_state
File "/Users/kevin/starkiller/research/airflow-pipelines/test_slack.py", line 5, in mystatehandler
SlackTask().run(message="test message", webhook_secret="SLACK_WEBHOOK_URL")
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/tasks.py", line 456, in method
return run_method(self, *args, **kwargs)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/tasks/notifications/slack_task.py", line 66, in run
r.raise_for_status()
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/requests/models.py", line 960, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: <https://hooks.slack.com/services/T015STTHK0A/B0342AESY6L/>
[2022-03-15 21:40:08-0400] INFO - prefect.TaskRunner | Task 'Add 1 task': Finished task run for task with final state: 'Failed'
[2022-03-15 21:40:08-0400] INFO - prefect.TaskRunner | Task '1/x task': Starting task run...
[2022-03-15 21:40:08-0400] ERROR - prefect.TaskRunner | Unexpected error while calling state handlers: HTTPError('403 Client Error: Forbidden for url: <https://hooks.slack.com/services/T015STTHK0A/B0342AESY6L/>')
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/runner.py", line 161, in handle_state_change
new_state = self.call_runner_target_handlers(old_state, new_state)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/task_runner.py", line 113, in call_runner_target_handlers
new_state = handler(self.task, old_state, new_state) or new_state
File "/Users/kevin/starkiller/research/airflow-pipelines/test_slack.py", line 5, in mystatehandler
SlackTask().run(message="test message", webhook_secret="SLACK_WEBHOOK_URL")
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/tasks.py", line 456, in method
return run_method(self, *args, **kwargs)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/tasks/notifications/slack_task.py", line 66, in run
r.raise_for_status()
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/requests/models.py", line 960, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: <https://hooks.slack.com/services/T015STTHK0A/B0342AESY6L/>
[2022-03-15 21:40:08-0400] INFO - prefect.TaskRunner | Task '1/x task': Finished task run for task with final state: 'Failed'
[2022-03-15 21:40:08-0400] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
(venv39) ➜ airflow-pipelines git:(minor_refactor) ✗
Kevin Kho
SLACK_WEBHOOK_URL = "<https://hooks.slack.com/services/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>"
and then i’ll tell you if I get the message. At that point we know the code is working fine hahaslack_notifier
because you can customize the messageKevin Otte
03/16/2022, 1:46 AMKevin Kho
Kevin Otte
03/16/2022, 1:46 AM(venv39) ➜ airflow-pipelines git:(minor_refactor) ✗ python test_slack.py
[2022-03-15 21:45:43-0400] INFO - prefect.FlowRunner | Beginning Flow run for 'Add Divide Flow'
[2022-03-15 21:45:43-0400] INFO - prefect.TaskRunner | Task 'Add 1 task': Starting task run...
[2022-03-15 21:45:43-0400] ERROR - prefect.TaskRunner | Unexpected error while calling state handlers: ValueError('Local Secret "<https://hooks.slack.com/services/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>" was not found.')
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/runner.py", line 161, in handle_state_change
new_state = self.call_runner_target_handlers(old_state, new_state)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/task_runner.py", line 113, in call_runner_target_handlers
new_state = handler(self.task, old_state, new_state) or new_state
File "/Users/kevin/starkiller/research/airflow-pipelines/test_slack.py", line 5, in mystatehandler
SlackTask().run(message="test message", webhook_secret= "<https://hooks.slack.com/services/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>")
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/tasks.py", line 456, in method
return run_method(self, *args, **kwargs)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/tasks/notifications/slack_task.py", line 61, in run
webhook_url = webhook_url or cast(str, Secret(webhook_secret).get())
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/client/secrets.py", line 167, in get
raise ValueError(
ValueError: Local Secret "<https://hooks.slack.com/services/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>" was not found.
[2022-03-15 21:45:43-0400] INFO - prefect.TaskRunner | Task 'Add 1 task': Finished task run for task with final state: 'Failed'
[2022-03-15 21:45:43-0400] INFO - prefect.TaskRunner | Task '1/x task': Starting task run...
[2022-03-15 21:45:43-0400] ERROR - prefect.TaskRunner | Unexpected error while calling state handlers: ValueError('Local Secret "<https://hooks.slack.com/services/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>" was not found.')
Traceback (most recent call last):
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/runner.py", line 161, in handle_state_change
new_state = self.call_runner_target_handlers(old_state, new_state)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/engine/task_runner.py", line 113, in call_runner_target_handlers
new_state = handler(self.task, old_state, new_state) or new_state
File "/Users/kevin/starkiller/research/airflow-pipelines/test_slack.py", line 5, in mystatehandler
SlackTask().run(message="test message", webhook_secret= "<https://hooks.slack.com/services/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>")
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/utilities/tasks.py", line 456, in method
return run_method(self, *args, **kwargs)
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/tasks/notifications/slack_task.py", line 61, in run
webhook_url = webhook_url or cast(str, Secret(webhook_secret).get())
File "/Users/kevin/starkiller/research/airflow-pipelines/venv39/lib/python3.9/site-packages/prefect/client/secrets.py", line 167, in get
raise ValueError(
ValueError: Local Secret "<https://hooks.slack.com/services/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>" was not found.
[2022-03-15 21:45:43-0400] INFO - prefect.TaskRunner | Task '1/x task': Finished task run for task with final state: 'Failed'
[2022-03-15 21:45:43-0400] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
(venv39) ➜ airflow-pipelines git:(minor_refactor) ✗
Kevin Kho
SlackTask
takes in the webhook_secret
which is a secret name. So you should be changing your secret in the config.toml and in the code still have`webhook_secret="SLACK_WEBHOOK_URL"`. You shouldn’t pass the webhook in there.Kevin Otte
03/16/2022, 1:50 AMKevin Kho