Aniruddha Sengupta
08/16/2022, 4:05 PMcompetitor
, lang_code
, model
and work_dir
@task(name="test_eval", log_stdout=True)
def run_test(
competitor: str,
lang_code: str,
testset_config: dict,
model: str,
work_dir: str
):
testset_path = testset_config[lang_code]
savepath = f"/exp/aniruddhas/comp_eval/test_outputs/2022-08-16/test_dynamic_outputs/{competitor}_{testset_path}_{lang_code}_{model}.txt"
return f'aladdin/evaluation/test.sh --competitor {competitor} --testset_path "{testset_path}" --lang_code {lang_code} --model "{model}" --work_dir "{work_dir}" | tee {savepath}'
From here, I have a list of competitors and a list of languages. I want to be able to create a Shell Prefect task but mapped over a list of competitors and languages. So far I have this:
shell_result = shell_task(
command=run_test.map(
competitors,
languages,
testset_config=unmapped(testset_config),
model=unmapped(model),
work_dir=unmapped(work_dir)
)
)
But this doesnt seem to work. I want a ShellTask to be able to iterate through each language for each competitor. How can I write this?competitors = [1, 2, 3]
languages = [en, fr, de]
Essentially I want to do the Prefect way of the following:
for competitor in competitors:
for lang in languages:
execute_task(competitor, lang)
Anna Geller
08/18/2022, 3:04 PMshell = ShellTask()
with Flow(...) as flow:
shell.map(...)