https://prefect.io logo
p

psimakis

10/01/2020, 11:28 AM
Hello everyone! I'm trying to find the right way to send a single slack notification for a group of mapped shell tasks. In my case, the list of mapped task is quite big, therefore, the slack messages are numerous and the slack channel has become really spammy. To be more specific, let's suppose that a group of similar shell commands should be mapped:
Copy code
from prefect.tasks.shell import ShellTask
from prefect import Flow

a = ShellTask()
commands = [
	'ls',
	'ls -l',
	'invalidcommand'
]

with Flow('test') as flow:
	b = a.map(commands)
	# send a single slack nofiication
	# that summarize the states of mapped tasks
	send_slack_notification()
The purpose of this slack notification is to summarize the states of mapped tasks by displaying the percentage of successfully mapped tasks. In the case above, this percentage will be 66.6% (the last command will fail). I tried to approach the problem using triggers and state handlers but I couldn't find a clean way to achieve the goal. Have you been in this situation before? Any hint? Thanks in advance!
n

nicholas

10/01/2020, 1:55 PM
Hi @psimakis! I think this is most neatly solved using the inherent reduction that Prefect does on mapped tasks. So in this case, you can store the result of your shell commands mapping in var
b
, and then pass that result to your
send_slack_notification
task.
Copy code
from prefect.tasks.shell import ShellTask
from prefect import Flow

#... other stuff

with Flow('test') as flow:
	b = a.map(commands)
	# send a single slack notification
	# that summarize the states of mapped tasks
	send_slack_notification(b)