psimakis
10/16/2020, 3:18 PMwebhook_secret
on a SlackTask
using a PrefectSecret
? I have tried to provide the webhook_secret
on the instantiation but I get a TypeError
Given this flow:
with Flow(name='flow name') as flow:
slack_custom_webhook_url = PrefectSecret('SLACK_E_SPIDER_STATISTICS')
spider_name = Parameter('spider_name', default='spider a')
with case(spider_name, 'spider a'):
SlackTask(webhook_secret=slack_custom_webhook_url)(message='Test')
Check out the full trace-back in the thread:nicholas
Alexander
10/16/2020, 4:15 PMSlackTask()(message='Test', webhook_secret=slack_custom_webhook_url)
Try thispsimakis
10/16/2020, 4:33 PMUnexpected error: TypeError('Object of type PrefectSecret is not JSON serializable')
Traceback (most recent call last):
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/client/secrets.py", line 137, in get
value = secrets[self.name]
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/box/box.py", line 471, in __getitem__
raise BoxKeyError(str(err)) from None
box.exceptions.BoxKeyError: '<Task: SLACK_E_SPIDER_STATISTICS>'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 825, in get_task_run_state
self.task.run, timeout=self.task.timeout, **raw_inputs
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/utilities/executors.py", line 188, in timeout_handler
return fn(*args, **kwargs)
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 449, in method
return run_method(self, *args, **kwargs)
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/tasks/notifications/slack_task.py", line 51, in run
webhook_url = cast(str, Secret(self.webhook_secret).get())
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/client/secrets.py", line 147, in get
variables=dict(name=self.name),
File "/home/user/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/client/client.py", line 279, in graphql
params=dict(query=parse_graphql(query), variables=json.dumps(variables)),
File "/usr/local/lib/python3.7/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/local/lib/python3.7/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.7/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/local/lib/python3.7/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type PrefectSecret is not JSON serializable
@Alexander well SlackTask.run
doesn't take webhook_secret
as argument: TypeError: got an unexpected keyword argument 'webhook_secret'
nicholas
SlackTask
in your own task and return its instantiation, so something like this:
Class DynamicSlackTask(SlackTask):
def run(self, webhook_url, message):
self.webhook_url = webhook_url
super(DynamicSlackTask).run(message=message)
with Flow(name='flow name') as flow:
slack_custom_webhook_url = PrefectSecret('SLACK_E_SPIDER_STATISTICS')
spider_name = Parameter('spider_name', default='spider a')
with case(spider_name, 'spider a'):
DynamicSlackTask()(webhook_secret=slack_custom_webhook_url, message='Test')
psimakis
10/16/2020, 4:50 PMnicholas
psimakis
10/16/2020, 5:10 PMnicholas
psimakis
10/16/2020, 11:00 PMself
should be provided in `super`: super(DynamicSlackTask, self).run(message=message)
. Thank you!nicholas