Hi. Newbie here. Was checking fitment of prefect ...
# prefect-communityw
Wing Rider
07/27/2020, 9:06 PMHi. Newbie here. Was checking fitment of prefect for a etl project. Can we trigger prefect flow based on condition. Say, when a file arrives ? (like a file polling service)
d
Dylan
07/27/2020, 9:11 PM
Hey @Wing Rider!
You may find this article helpful: https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a
💥 2
Dylan
07/27/2020, 9:12 PM
Shorter answer: right now there are two ways to kick off a flow run with Prefect Cloud
1. with an http request
2. on a schedule
👍 1
Dylan
07/27/2020, 9:12 PM
There are a bunch of great event-driven serverless products on the market that will let you make an HTTP request to Cloud/Server based on an event. Here’s an example on how to use one of them!
w
Wing Rider
10/14/2020, 10:50 AMHi @Dylan, would you able to give me an http post request sample.
d
Dylan
10/14/2020, 2:37 PM
Dylan
10/14/2020, 2:37 PM
Feel free to use the python client above ☝️
Dylan
10/14/2020, 2:37 PM
Otherwise, here’s the GraphQL: https://docs.prefect.io/orchestration/concepts/flow_runs.html#graphql
w
Wing Rider
10/14/2020, 9:30 PMThanks @Dylan, we are trying to invoke directly from Azure logic apps as a direct post request. For that we need to give post body and headers.
I reckon based on the articles the structure of the post request should be below.
body
mutation {create_flow_run(input: {flow_id: "<flow id>",parameters: "{\"a\":2}"}) {id}}1) "Content-Type", "application/json"2) "Authorization", "Bearer {}".format("<authtoken>"d
Dylan
10/14/2020, 9:33 PM
Yup! That’s correct
👍 1
w
Wing Rider
10/14/2020, 9:44 PMwill let you know how it goes
Wing Rider
10/15/2020, 1:19 AMtried to do an post request. our api is https://api.prefect.io/graphql
Body
mutation {create_flow_run(input: { flow_id: "e00006c7-e24c-480b-b971-0dc90f80668e" }) {id}}{"Authorization": "*sanitized*","Content-Type": "application/json"}<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><title>Error</title></head><body><pre>Bad Request</pre></body></html>Wing Rider
10/15/2020, 1:56 AMThe body is working fine from the interactive api
Wing Rider
10/15/2020, 3:43 AMi believe the problem is with Azure logic apps.
Wing Rider
10/15/2020, 3:43 AMI am able to invoke the flow from postman
Wing Rider
10/15/2020, 3:44 AMi will try from a python client and see if we can invoke it
d
Dylan
10/15/2020, 3:55 PM
Hey Wing how did this go?
Dylan
10/15/2020, 3:55 PM
Dylan
10/15/2020, 3:55 PM
Here’s the client code if it’s helpful
w
Wing Rider
10/15/2020, 9:50 PMThanks alot @Dylan. It went fine. The code from the blog itself is (https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a) is working very fine. Instead of lambda we exposed it in Azure functions as we are in Azure platform. Logic apps just invokes the functions
d
Dylan
10/15/2020, 9:51 PM
Awesome!
Dylan
10/15/2020, 9:51 PM
Glad I could help 😄
w
Wing Rider
10/15/2020, 9:56 PMI have got a slack notifier question. Do I need to start a different thread?
d
Dylan
10/15/2020, 9:57 PM
I’m happy to help here
w
Wing Rider
10/15/2020, 9:57 PMWe need to post custom messages to the slack
Wing Rider
10/15/2020, 9:57 PMslack_notifier as a state handler is working fine for the states
Wing Rider
10/15/2020, 9:58 PMbut as we are running mapped commands, we need to post the variables in slack to understand which variable is executing now and errored
Wing Rider
10/15/2020, 9:58 PMfor instance , startup parameters needs to be printed
Wing Rider
10/15/2020, 9:59 PM Copy code
<http://logger.info|logger.info>(f"{response}")
slack_notifier(f"{response}")Wing Rider
10/15/2020, 9:59 PMTried this way. Logger is working fine. but slack_notifier didnt post anything even though it didnt throw any error for the above usage
Wing Rider
10/15/2020, 10:06 PMdo we need to do a context set similar to logger for slack as well?
Wing Rider
10/15/2020, 10:06 PMlogger = prefect.context.get("logger")d
Dylan
10/15/2020, 10:18 PM
Yes most likely
Dylan
10/15/2020, 10:19 PM
I believe there’s an issue to customize slack messages
Dylan
10/15/2020, 10:19 PM
via the notifier
w
Wing Rider
10/15/2020, 10:53 PMis it. we need atleast the flowid to be displayed to understand which flow got completed
Wing Rider
10/15/2020, 11:21 PMif you can giveme a sample , it would be helpful