Hi All. I have a dumb design question on how to use Prefect cloud + ECS Fargate. I have a flow registered to an ECS cluster. Now I want to iterate over a list of value for a parameter(e.g.[1,2,3,4,5]) that I used inside a task or the registered flow:
example:
@task
def make_df(i):
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("Hi from Prefect %s", prefect.__version__)
<http://logger.info|logger.info>("this is the first step")
data = {'Name':['Tom', 'Brad', 'Kyle', 'Jerry'],
'Age':[20, i**2, 2*i, 18*i],
'Height' : [6.1, 5.9, 6.0, 6.1]
}
df = pd.DataFrame(data)
return df
How should I design my work to run 5 parallel task in ECS cluster?