Hi folks In prefect `0 13 1` I can t get the `slack notifier

Question

Hi folks In prefect `0 13 1` I can t get the `slack notifier` to work This used to work in `0 12 1` Here is a minimal example of a flow ```from prefect import Flow task context from prefect utilities notifications import slack notifier task def log run id context get logger warning context get flow run id task state handlers= slack notifier def get 1 return 1 with Flow slack example as flow log = log run id result = get 1 upstream tasks= log flow run ``` Apparently `slack notifier` attempts to communicate with a backend server but I am not going to use one More info in thread

emre · Answer

Complete logs in case anyone needs

emre · Answer

``` 2020 08 11 07 47 52 INFO prefect FlowRunner | Beginning Flow run for slack example 2020 08 11 07 47 52 INFO prefect FlowRunner | Starting flow run 2020 08 11 07 47 52 INFO prefect TaskRunner | Task log run id Starting task run 2020 08 11 07 47 52 WARNING prefect log run id | d36c58d1 4318 4a22 a317 bf7b79e38fd8 2020 08 11 07 47 52 INFO prefect TaskRunner | Task log run id finished task run for task with final state Success 2020 08 11 07 47 52 INFO prefect TaskRunner | Task get 1 Starting task run 2020 08 11 07 47 53 ERROR prefect TaskRunner | Unexpected error while calling state handlers ConnectionError MaxRetryError HTTPConnectionPool host= localhost port=4200 Max retries exceeded with url graphql Caused by NewConnectionError lt urllib3 connection HTTPConnection object at 0x7f9a2742de50 gt Failed to establish a new connection Errno 61 Connection refused Traceback most recent call last File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 connection py line 159 in new conn conn = connection create connection File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 util connection py line 84 in create connection raise err File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 util connection py line 74 in create connection sock connect sa ConnectionRefusedError Errno 61 Connection refused During handling of the above exception another exception occurred Traceback most recent call last File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 connectionpool py line 670 in urlopen httplib response = self make request File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 connectionpool py line 392 in make request conn request method url httplib request kw File Library Frameworks Python framework Versions 3 8 lib python3 8 http client py line 1230 in request self send request method url body headers encode chunked File Library Frameworks Python framework Versions 3 8 lib python3 8 http client py line 1276 in send request self endheaders body encode chunked=encode chunked File Library Frameworks Python framework Versions 3 8 lib python3 8 http client py line 1225 in endheaders self send output message body encode chunked=encode chunked File Library Frameworks Python framework Versions 3 8 lib python3 8 http client py line 1004 in send output self send msg File Library Frameworks Python framework Versions 3 8 lib python3 8 http client py line 944 in send self connect File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 connection py line 187 in connect conn = self new conn File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 connection py line 171 in new conn raise NewConnectionError urllib3 exceptions NewConnectionError lt urllib3 connection HTTPConnection object at 0x7f9a2742de50 gt Failed to establish a new connection Errno 61 Connection refused During handling of the above exception another exception occurred Traceback most recent call last File Users emreakgun Documents prefect server venv lib python3 8 site packages requests adapters py line 439 in send resp = conn urlopen File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 connectionpool py line 724 in urlopen retries = retries increment File Users emreakgun Documents prefect server venv lib python3 8 site packages urllib3 util retry py line 439 in increment raise MaxRetryError pool url error or ResponseError cause urllib3 exceptions MaxRetryError HTTPConnectionPool host= localhost port=4200 Max retries exceeded with url graphql Caused by NewConnectionError lt urllib3 connection HTTPConnection object at 0x7f9a2742de50 gt Failed to establish a new connection Errno 61 Connection refused During handling of the above exception another exception occurred Traceback most recent call last File Users emreakgun Documents prefect server prefect src prefect engine runner py line 162 in handle state change new state = self call runner target handlers old state new state File Users emreakgun Documents prefect server prefect src prefect engine task runner py line 111 in call runner target handlers new state = handler self task old state new state or new state File Users emreakgun Documents prefect server venv lib python3 8 site packages toolz functoolz py line 303 in call return self partial args kwargs File Users emreakgun Documents prefect server prefect src prefect utilities notifications notifications py line 305 in slack notifier form data = slack message formatter tracked obj new state File Users emreakgun Documents prefect server prefect src prefect utilities notifications notifications py line 159 in slack message formatter url = prefect client Client get cloud url File Users emreakgun Documents prefect server prefect src prefect client client py line 117 in init tenant info = self graphql query tenant id File Users emreakgun Documents prefect server prefect src prefect client client py line 268 in graphql result = File Users emreakgun Documents prefect server prefect src prefect client client py line 223 in post response = self request File Users emreakgun Documents prefect server prefect src prefect client client py line 393 in request response = self send request File Users emreakgun Documents prefect server prefect src prefect client client py line 312 in send request response = url headers=headers json=params timeout=30 File Users emreakgun Documents prefect server venv lib python3 8 site packages requests sessions py line 578 in post return self request POST url data=data json=json kwargs File Users emreakgun Documents prefect server venv lib python3 8 site packages requests sessions py line 530 in request resp = self send prep send kwargs File Users emreakgun Documents prefect server venv lib python3 8 site packages requests sessions py line 643 in send r = adapter send request kwargs File Users emreakgun Documents prefect server venv lib python3 8 site packages requests adapters py line 516 in send raise ConnectionError e request=request requests exceptions ConnectionError HTTPConnectionPool host= localhost port=4200 Max retries exceeded with url graphql Caused by NewConnectionError lt urllib3 connection HTTPConnection object at 0x7f9a2742de50 gt Failed to establish a new connection Errno 61 Connection refused 2020 08 11 07 47 53 INFO prefect TaskRunner | Task get 1 finished task run for task with final state Failed 2020 08 11 07 47 53 INFO prefect FlowRunner | Flow run FAILED some reference tasks failed ```

emre · Answer

As far as I understand if a flow run is connected to a backend a `flow run id` is present in `prefect context` Through the existence of `flow run id` `slack notifier` knows that it should ask the backend for a relevant url and supply the notification with said url In `0 13 1` a `flow run id` is available even without a backend therefore `slack notifier` gets a Connection Refused error Line 151 in `slack notifier` ```if prefect context get flow run id url = None if isinstance tracked obj prefect Flow url = prefect client Client get cloud url flow run prefect context flow run id as user=False elif isinstance tracked obj prefect Task url = prefect client Client get cloud url task run prefect context get task run id as user=False if url notification payload update title link=url ```

emmanuel · Answer

You could probably override the class to fetch the web hook from somewhere else couldn t you

emmanuel · Answer

h mm it s actually just a function really

emmanuel · Answer

``` webhook secret str optional the name of the Prefect Secret that stores your slack webhook URL defaults to ` SLACK WEBHOOK URL ```` maybe you can simply inject it with an environment variable in which case Prefect won t try to fetch it from a remote location

emmanuel · Answer

``` webhook url = cast str prefect client Secret webhook secret or SLACK WEBHOOK URL get ``` nope smile

emmanuel · Answer

Well I guess you could write your own notification function which would work just the same but take the webhook url from somewhere else

emre · Answer

Sure I could always write my own Still it is a functionality supplied by prefect that used to work but now doesn t Felt like something to report

emmanuel · Answer

how did you provide the url in `0 12 1`

emmanuel · Answer

out of curiosity

emre · Answer

Thats the point I did not The prefect client uses a default depending on your backend choice

emre · Answer

And I believe a default url always exists

emre · Answer

Nothing changed with `slack notifier` But starting from `0 12 2` flow objects started to supply a `flow run id` to their `flow run context` Search for `flow run id` here and then do the same in

emmanuel · Answer

weird because Slack web hook are customer specific <https api slack com messaging webhooks> thinking face

emre · Answer

prefect slack notifier used `flow run id` to distinguish core only runs and runs backed by a server but this starting from `0 12 2` this separation doesn t work anymore

emre · Answer

Oh you mean slack webhook Sure it is an env var secret in my example I was talking about prefect server url

emre · Answer

The connection issue isn t to slack it is to prefect server My flow run is without a backend server but the notifier attempts to connect to a prefect backend regardless

emmanuel · Answer

gt The connection issue isn t to slack it is to prefect server Yeah I got that It tries to connect to the graphql server endpoint to retrieve the slack endpoint to use right At least that s what I understand reading the `slack notifier` code

emre · Answer

Exactly

emre · Answer

The check for whether the flow run is backed by a server or standalone is old it is not valid anymore

emmanuel · Answer

h mmm it looks like the notifier code is the same in `0 12 x` <https github com PrefectHQ prefect blob 0 12 1 src prefect utilities notifications notifications py L278> so probably it s a change in the client class confused

emmanuel · Answer

what s the name of the env var you use `SLACK WEBHOOK URL`

emmanuel · Answer

according to the doc it should probably be something like that now `PREFECT CONTEXT SECRETS SLACK WEBHOOK URL`

emmanuel · Answer

because of this <https github com PrefectHQ prefect blob master src prefect client secrets py L135>

emmanuel · Answer

well that s my understanding only confused

emre · Answer

yeah my variable is name `PREFECT CONTEXT SECRETS SLACK WEBHOOK URL` I think I understood the issue will open a github issue

emre · Answer

Ben Davison · Answer

If your on `0 13 1` could you use the new webhook feature I haven t tried it yet

emre · Answer

Doesn t that require a server cloud I am going to run this standalone

Ben Davison · Answer

Docs don t seem to be updated yet You can see webhooks are part of the new release