https://prefect.io logo
Title
w

Wing Rider

07/27/2020, 9:06 PM
Hi. Newbie here. Was checking fitment of prefect for a etl project. Can we trigger prefect flow based on condition. Say, when a file arrives ? (like a file polling service)
d

Dylan

07/27/2020, 9:11 PM
Hey @Wing Rider! You may find this article helpful: https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a
💥 2
Shorter answer: right now there are two ways to kick off a flow run with Prefect Cloud 1. with an http request 2. on a schedule
👍 1
There are a bunch of great event-driven serverless products on the market that will let you make an HTTP request to Cloud/Server based on an event. Here’s an example on how to use one of them!
w

Wing Rider

10/14/2020, 10:50 AM
Hi @Dylan, would you able to give me an http post request sample.
Feel free to use the python client above ☝️
w

Wing Rider

10/14/2020, 9:30 PM
Thanks @Dylan, we are trying to invoke directly from Azure logic apps as a direct post request. For that we need to give post body and headers. I reckon based on the articles the structure of the post request should be below. body
mutation {
create_flow_run(input: {
flow_id: "<flow id>",
parameters: "{\"a\":2}"
}) {
id
}
}
Headers
1) "Content-Type", "application/json"
2) "Authorization", "Bearer {}".format("<authtoken>"
d

Dylan

10/14/2020, 9:33 PM
Yup! That’s correct
👍 1
w

Wing Rider

10/14/2020, 9:44 PM
will let you know how it goes
tried to do an post request. our api is https://api.prefect.io/graphql Body
mutation {
create_flow_run(input: { flow_id: "e00006c7-e24c-480b-b971-0dc90f80668e" }) {
id
}
}
Headers
{
  
"Authorization": "*sanitized*",
  
"Content-Type": "application/json"
}
Response - Status 400
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Bad Request</pre>
</body>
</html>
The body is working fine from the interactive api
i believe the problem is with Azure logic apps.
I am able to invoke the flow from postman
i will try from a python client and see if we can invoke it
d

Dylan

10/15/2020, 3:55 PM
Hey Wing how did this go?
Here’s the client code if it’s helpful
w

Wing Rider

10/15/2020, 9:50 PM
Thanks alot @Dylan. It went fine. The code from the blog itself is (https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a) is working very fine. Instead of lambda we exposed it in Azure functions as we are in Azure platform. Logic apps just invokes the functions
d

Dylan

10/15/2020, 9:51 PM
Awesome!
Glad I could help 😄
w

Wing Rider

10/15/2020, 9:56 PM
I have got a slack notifier question. Do I need to start a different thread?
d

Dylan

10/15/2020, 9:57 PM
I’m happy to help here
w

Wing Rider

10/15/2020, 9:57 PM
We need to post custom messages to the slack
slack_notifier as a state handler is working fine for the states
but as we are running mapped commands, we need to post the variables in slack to understand which variable is executing now and errored
for instance , startup parameters needs to be printed
<http://logger.info|logger.info>(f"{response}")
slack_notifier(f"{response}")
Tried this way. Logger is working fine. but slack_notifier didnt post anything even though it didnt throw any error for the above usage
do we need to do a context set similar to logger for slack as well?
logger = prefect.context.get("logger")
d

Dylan

10/15/2020, 10:18 PM
Yes most likely
I believe there’s an issue to customize slack messages
via the notifier
w

Wing Rider

10/15/2020, 10:53 PM
is it. we need atleast the flowid to be displayed to understand which flow got completed
if you can giveme a sample , it would be helpful