William Clark
11/29/2021, 5:28 PMKevin Kho
with Flow(...) as flow:
x = Parameter("x", True)
with case(x, True):
create_flow_run(...)
and you can parameterize the start of sub flow runs this way. Is this what you mean?William Clark
11/29/2021, 5:40 PMWilliam Clark
11/29/2021, 5:40 PMbucket_1 = 'bucket'
model_1 = 'cool_model_1'
model_2 = 'cool_model_2'
@task('Get Model')
def get_model_s3(bucket_name, model_name) -> Path:
"Gets a trained model from S3 and stores it in a temporary directory"
return Path(f'/tmp/{model_name}')
@task('Evaluate Model')
def evaluate_model(model):
"Evaluates a model"
return model.evaluate()
with Flow('Evalulate Model'):
bucket_name = Parameter('bucket_name')
model_name = Parameter('model_name')
model = get_model_s3(bucket_name, model_name)
evaluate_model(model)
flow.run()
William Clark
11/29/2021, 5:43 PMWilliam Clark
11/29/2021, 5:45 PMWilliam Clark
11/29/2021, 5:46 PMKevin Kho
@task
def make_param_dict(model_name):
return {"model_name": model_name}
with Flow(...) as flow:
models = Parameter("models", ["1","2","3"])
param_dict = make_param_dict.map(models)
create_flow_run.map(..., parameters=param_dict)
William Clark
11/29/2021, 10:01 PMKevin Kho
William Clark
11/29/2021, 11:18 PMKevin Kho
.map()
can have multiple parameters per task to answer your question. And of course! No problem!