https://prefect.io logo
j

Jasono

11/02/2020, 3:17 AM
Hi I’m trying to define tasks imperatively based on a list. The challenge is that the list should be based on a Prefect parameter. Below is the stackoverflow link to the code I tried, but apparently it doesn’t work as task_dependency_pairs is a task, not a list. How do I make it work without breaking the dependency between the Parameter task, and the other dynamically generated tasks?
s

Spencer

11/02/2020, 3:40 AM
It seems you're trying to use a runtime detail (the task output of
task_dependency_pairs
) to essentially create another flow dynamically at runtime. Is that correct?
j

Jasono

11/02/2020, 3:42 AM
dynamically creating tasks, yes
s

Spencer

11/02/2020, 3:44 AM
Dynamically creating tasks is one thing, but it appears that you're also dynamically setting upstream tasks (essentially a whole flow)
Dynamically creating tasks is typically handled with the mapping functionality: https://docs.prefect.io/core/concepts/mapping.html
j

Jasono

11/02/2020, 4:13 AM
Thank you!
3 Views