https://prefect.io
Join Slack
We have a requirement to create a Prefect `Cloud Hook` for the Flow `success` event. This needs to ...
# ask-community
s
We have a requirement to create a Prefect 
Cloud Hook
for the Flow 
success
event. This needs to be done programmatically via the api at registration time. The documentation states 
Psst! We have recently added Automations which offer more functionality than Cloud Hooks and will eventually replace them.
 There does not seem to currently be programmatic api access to Automations. 1. Is there a way to register an Automation via an API and if so is it documented somewhere? 2. When is the planned deprecation of Cloud Hooks? We will need to support a mix of self hosted Prefect Server instances and Prefect Cloud usage so we would like a solution which supports both but it appears that Automations will not be available via Prefect Server.
j
Hi @Sean Harkins - you can use the API for automations - the best place for documentation is in the Interactive API or there are some detailed questions and answers here in slack. I'm going to make a note that we should add more docs for them too. For your second question - let me check if we have a deprecation date for cloud hooks.
s
Thanks @Jenny 👍 Will Automations be available in our self-hosted Prefect Server instances or only through Prefect Cloud?
j
You're welcome @Sean Harkins Automations are only available in Prefect Cloud. Let us know if you want to use the api for automations and have any questions!
Hi @Sean Harkins - update for you that we won't be adding anything new to them but there's no date for ending use of cloud hooks.
s
Thanks @Jenny. In that case I may be sticking with Cloud Hooks to maintain portability between Prefect Cloud and Prefect Server. I have been tracking some of the posts from @Mark McDonald here on Slack but I may ping you with some questions as I implement things. Cheers.
j
Sounds good!
One other option, if you just want notification of a flow run success state, would be to use a state handler: https://docs.prefect.io/core/concepts/notifications.html#state-handlers
s
@Jenny It looks like Automations might be the path we need to take for now as we need to manipulate the webhook payload and headers. Is there a way to manually trigger an automation via the UI for testing purposes as there is for webhooks?
j
Hi Sean - I was out of the office yesterday sorry this got missed. There's currently no test from the UI but you can use the API and the following mutation:
Copy code
mutation {
  test_action (input: {action_id: ""}){
    success
  }
}
s
👍 Thanks @Jenny
@Jenny Hopefully last question. I’m now trying to diagnose my webhook not being triggered by an automation. Is there a way to query automation executions so I can view the payload which is being sent?
j
@Sean Harkins - my best tip here is to check the notifications tile in cloud as we send a notification if there's a configuration error with an action.
Does the webhook work if you use the test_action mutation?
s
👍 @Jenny The issue was on the consumer side. I had previously registered the hook using a development Github token and it needed to be updated.
j
Ah ok. Does that mean you have it working now?
s
👍 Yup. All set. You can view the finalized automation creation code here https://github.com/pangeo-forge/pangeo-forge-prefect/blob/master/pangeo_forge_prefect/automation_hook_manager.py
Thanks for the assistance.
🚀 1
j
Great to hear. And thanks for sharing!
s
@Jenny Sorry to ping you with more questions 😄 What would the GraphQL syntax look like for deleting an automation? Currently we just ignore the existing Automation but it looks like we are going to need to replace it to support another business case. You can see the code we are using currently in the link above ☝️
Browsing the API I see 
create_flow_run_state_changed_hook
but nothing for deleting or updating.
j
Hi @Sean Harkins - automations are made up of hooks and actions - you can use the 
delete_hook
and 
delete_action
mutations. I'm about to head into a meeting but I can send an example in a bit.
s
That would be awesome. Thanks @Jenny
j
Ok! To remove a hook you can find its ID and the connected actions (and their ID) using:
Copy code
query {
  hook {
    id
    event_tags
    action {
      id
    }
    created
  }
}
Then to delete it:
Copy code
mutation {
  delete_hook (input: {hook_id: "12dd1e90-46b4-4962-a142-272603b454f6"}) {
    success
  }
}
(Hooks are basically the API for automations.)
Then to delete its connected actions:
Copy code
mutation {
  delete_action (input: {action_id: "811add75-9b76-4228-aee6-87ba94d6d253"}) {
    success
  }
}
Of course, replace those IDs with what you get returned from your query.
s
@Jenny Can you explain how I would include the action id property as output in my query via Python?
Copy code
hook_query = {
        "query": {
            with_args(
                "hook", {"where": {"event_tags": {"_contains": {"flow_group_id": [flow_group_id]}}}}
            ): {
                "id"
            }
        }
    }
Trying to include “action”: {“id”} complains as invalid.
👀 1
j
Hi Sean, I don't usually use that with_args so not sure how to do it that way. I'd write my query (with apologies for formatting) as:
Copy code
query = """
    query {
  hook {
    id
    event_tags
    action {
      id
    }
  }
}
"""
You could also use variable to filter:
Copy code
query = """
query ($etContains: jsonb){
  hook (where: {event_tags: {_contains: $etContains}}) {
    id
    event_tags
    action {
      id
    }
  }
}
"""

variables={'etContains': 'xxxxx'}

client.graphql(
    query, variables=variables
)
But heads up that I found that query filtered out all my hooks so you may be better leaving that out and do a post query filter.
4 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by