Hi
I want to create multiple task instances with different credentials to get data from several accounts
Copy code
with Flow("import") as flow:
for i in range(5):
res = import_task(
command="echo $CREDENTIALS",
env={
"CREDENTIALS": str(i)
},
)
But in that case all tasks will run in parallel, which will cause the request limits to be exceeded
Can I run those tasks sequentially ?
a
Anna Geller
06/07/2022, 11:59 AM
Yes - by default your flow uses LocalExecutor and everything runs sequentially
You shouldn't use a for loop in a Prefect 1.0 flow - you would need mapping for this
in Prefect 2.0, you can run for loops and arbitrary Python code though
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.