st dabu
with Flow("NLP") as nlp_flow: aws s3 cp blah blah sometool /data/a.csv
Kevin Kho
from prefect.utilities.tasks import task from prefect import task, Flow from prefect.tasks.shell import ShellTask @task def files(): return [ '/opt/file1.txt', '/opt/file2.txt', '/opt/file3.txt', '/opt/file4.txt', ] @task def format_commands(file_name): return f'echo "{file_name}"' rm_task = ShellTask() with Flow('shell') as flow: files_to_delete = files() commands = format_commands.map(files_to_delete) rm_task.map(command=commands) flow.run()
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.