Dekel R

    Dekel R

    4 months ago
    Hey all, 2 flows that are running everyday for the last couple of months returned this error yesterday:
    Exception raised while calling state handlers: ClientError([{'path': ['secret_value'], 'message': 'An unknown error occurred.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}])
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/site-packages/prefect/client/secrets.py", line 137, in get
        value = secrets[self.name]
    KeyError: 'SLACK_WEBHOOK_URL'
    I get this error randomly once in a while - anyone have the same issue? I didn’t reregister or touched the 2 flows for at least 2 months now. Thanks
    Anna Geller

    Anna Geller

    4 months ago
    interesting, are you on Prefect Cloud or Server? could you show how you use this Secret task in your flow? sometimes when you use it with mapping, you may end up retrieving the secret separately for each task and making multiple requests simultaneously and some of those requests can fail - passing the secret as data dependency or attaching the secret to storage may help solve the issue
    Dekel R

    Dekel R

    4 months ago
    Hey, I’m using Prefect cloud. I don’t get this secret explicitly but it get’s used by Prefect state handler -
    handler = slack_notifier(only_states=[Failed])
    
    
    @task(state_handlers=[handler])
    def get_relevant_dates(param_date: str) -> datetime.date:
        logger = prefect.context.get(LOGGER_NAME)
        if param_date is None:
            param_date = pendulum.yesterday(tz="America/New_York").subtract(days=1).date()
        else:
            try:
                param_date = datetime.strptime(param_date, '%d-%m-%y').date()
            except Exception as e:
                logger.error(e)
        return param_date
    Anna Geller

    Anna Geller

    4 months ago
    I can't see any secret in the code you've just sent 😄
    Kevin Kho

    Kevin Kho

    4 months ago
    The secret is in the
    slack_notifier
    handler that they attached to the task
    If you are using this
    slack_notifier
    multiple times in the Flow, you can add the secret to storage
    flow.storage = Storage(…, secrets=["SLACK_WEBHOOK_URL"])
    This will pull the secret into context during the flow run so you don’t have to repeatedly pull
    Anna Geller

    Anna Geller

    4 months ago
    Kevin, you're right and I discussed with @Dekel R via DM that he may use
    SlackTask(message=msg).run()
    instead
    Dekel R

    Dekel R

    4 months ago
    Hey, Thanks for the help. I have a couple of questions:1. Is slack notifier deprecated? I used it after I read this documentation page https://docs.prefect.io/core/advanced_tutorials/slack-notifications.html. If its not deprecated I think it should still work (correct me if I’m wrong). 2. Whats the cause of this issue? it seems like Prefect cloud is not available when the notifier requests this secret - is it a bug? 3. Should I completely switch the way we handle notifications from slack_notifier to SlackTask(message=msg).run() ? Thanks
    Anna Geller

    Anna Geller

    4 months ago
    #1 not officially, but SlackTask and state handler are easier to use and recommended #2 likely frequent API calls - attaching secret to storage as Kevin showed may help you fix this #3 You may, yes, but it's up to you