Matthew Blau
02/02/2021, 5:00 PMdef post_to_slack(task, old_state, new_state):
my_secret = PrefectSecret("SLACK_WEBHOOK_URL").run()
if new_state.is_retrying():
msg = "Task {0} failed and is retrying at {1}".format(task, new_state.start_time)
# replace URL with your Slack webhook URL
<http://requests.post|requests.post>(my_secret, json={"text": msg})
elif new_state.is_failed():
msg = "Task {0} failed".format(task)
# replace URL with your Slack webhook URL
<http://requests.post|requests.post>(my_secret, json={"text": msg})
return new_state
I specifically receive this message if I leave the code as it is above:
Exception raised while calling state handlers: ClientError('400 Client Error: Bad Request for url: <http://host.docker.internal:4200/graphql>\n\nThe following error messages were provided by the GraphQL server:\n\n GRAPHQL_VALIDATION_FAILED: Cannot query field "secret_value" on type "Query".\n\nThe GraphQL query was:\n\n query($name: String!) {\n secret_value(name: $name)\n }\n\nThe passed variables were:\n\n {"name": "SLACK_WEBHOOK_URL"}\n')
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/client/secrets.py", line 137, in get
value = secrets[self.name]
KeyError: 'SLACK_WEBHOOK_URL'
Thank you in advance!Zanie
Matthew Blau
02/02/2021, 5:32 PMdef post_to_slack(task, old_state, new_state):
my_secret = Secret("SLACK_WEBHOOK_URL").get()
if new_state.is_retrying():
msg = "Task {0} failed and is retrying at {1}".format(task, new_state.start_time)
# replace URL with your Slack webhook URL
<http://requests.post|requests.post>(my_secret, json={"text": msg})
elif new_state.is_failed():
msg = "Task {0} failed".format(task)
# replace URL with your Slack webhook URL
<http://requests.post|requests.post>(my_secret, json={"text": msg})
return new_state
Am I missing something with how the secrets need to be handled?s = Secret("SLACK_WEBHOOK_URL") and then call s.get()
and it parses and prints out the proper value. So obviously the key exists and prefect can parse it. I am confused as to why it is unable to parse it in my post_to_slack() methodZanie
prefect backend server
? Can you give me the output of prefect diagnostics
try:
value = secrets[self.name]
except KeyError:
if prefect.config.backend != "cloud":
raise ValueError(
'Local Secret "{}" was not found.'.format(self.name)
) from None
Matthew Blau
02/02/2021, 8:37 PM{
"config_overrides": {
"context": {
"secrets": false
}
},
"env_vars": [],
"system_information": {
"platform": "Linux-5.4.0-65-generic-x86_64-with-glibc2.29",
"prefect_backend": "server",
"prefect_version": "0.14.6",
"python_version": "3.8.5"
}
}
[context.secrets]
SLACK_WEBHOOK_URL="url_here"
Zanie
Matthew Blau
02/02/2021, 9:04 PMZanie
DockerRun
config with an environment variable?
PREFECT__CONTEXT__SECRETS__SLACK_WEBHOOK_URL="foo"
Matthew Blau
02/02/2021, 9:23 PMwith Flow("example",
storage = Docker(dockerfile="/home/mblau/projects/experian-integration/Dockerfile", ignore_healthchecks= False,
)) as flow:
result = main()
what would you like for me to adjust here?Zanie
--env
(https://docs.prefect.io/api/latest/cli/agent.html#docker-start) or use a run config ie https://docs.prefect.io/orchestration/agents/docker.html#flow-configurationMatthew Blau
02/02/2021, 9:29 PMZanie
env
of a DockerRun
or generate the command to run the agent. The secrets aren't automatically passed through from a local context to the agent.Matthew Blau
02/02/2021, 9:37 PMpython3 integration.py
in order to have the flow be registered with the UI. It works and errors expectedly, however, the Slack Notification does not fire off the message of "Task: test slack failed" without that runtime arg being passed to the DockerAgentZanie
Matthew Blau
02/02/2021, 9:50 PMZanie
In [1]: from prefect import Flow
fr
In [2]: from prefect.run_configs import DockerRun
In [3]: with Flow("ex") as flow:
...: pass
...:
In [4]: from prefect import config
In [6]: flow.run_config = DockerRun(env={f"PREFECT__CONTEXT__SECRETS__{k}": v for k, v in config.context.secrets.items()})
In [7]: flow.run_config.env
Out[7]: {'PREFECT__CONTEXT__SECRETS__fooo': 'bar'}
Matthew Blau
02/03/2021, 2:33 PMZanie
Matthew Blau
02/03/2021, 3:32 PMZanie
Matthew Blau
02/03/2021, 7:11 PMZanie
new_state.result
variable. I would highly recommend putting a breakpoint()
in your state handler and examining what is available to you in the case you’re interested in.