https://prefect.io logo
Title
n

Nace Plesko

10/11/2022, 10:47 PM
Hi, I'm running Prefect V1 and I'm having problems with passing in parameter. I'm trying to pass in Parameter to flow and use that parameter to concat a
command
for
ShellTask
. But the problem is that I'm not using the
Parameter
directly and it's not showing up in the UI when I register the flow. Is there any way to get around it? Thank you in advance!
n

Nico Neumann

10/11/2022, 10:59 PM
In prefect v1 it is not possible to directly create the command string inside the flow. You need to wrap it inside a task which generates the string. I am not familiar with mapping the ShellTask, but would instead provide the command directly
@task
def create_command(param: str) -> str:
    return f"yarn ... --name {param}"

with Flow("My Flow") as flow:
    param = Parameter("parameter", default="nace")
    command = create_command(param)
    
    ShellTask(
        command=command,
        ...
    )
n

Nace Plesko

10/11/2022, 11:18 PM
Thanks, that worked! The only issue now is that I have multiple commands passed as a list into
ShellTask
and all of them still run even if the first one fails. Is there a way to configure
ShellTask
so that it stops as soon as one of the commands fails?
🙌 1
n

Nico Neumann

10/11/2022, 11:33 PM
So you are using map to run multiple ShellTasks at the same time and want to stop all of them directly when one of them fails?
n

Nace Plesko

10/11/2022, 11:40 PM
Almost, they are run in sequence
Right now .map is creating a new shelltask as soon as the previous shelltask from .map finished (it doesn't matter if it succeeded or not). I'm trying to configure it in such way that as soon as the first shelltask in that sequence fails, to stop the mapping and fail the flow
^ bump on this one. Is there a way to set it up so that the mapped commands run only if a previous command that was generated as part of the same
.map
ran successfully?
✅ 1
m

Mason Menges

10/19/2022, 3:54 PM
Hey @Nace Plesko you should be able to set an all successful trigger on the task which should require all upstream mapped tasks to be successful before they are triggered more on that here https://docs-v1.prefect.io/core/concepts/mapping.html#state-behavior-with-mapped-tasks
n

Nace Plesko

10/19/2022, 4:05 PM
I'll give this a try, thanks! I think I misunderstood how that works, but maybe it works in the way I want it to work!
Unfortunately that didn't work. Setting trigger works for upstream tasks, but not among the mapped tasks themselves. My mapped tasks depend on each other, they get correctly mapped one after another, but if the first one fails, the second one still runs. I can't find a way to prevent that