Related to my above question, how do I access the value of a parameter? For example, I want to use

flattened_lists = itertools.chain(*nested_lists_param)

but can't iterate through the Parameter itself.

Hi @Jeff Yun - remember that Prefect builds a deferred computation graph, so no code is executed until the Flow is actually run (which also means the values of tasks / parameters are only available at run-time). That being said, you should be able to do:

@task(name='create-product')
def create_product(params1, params2):
   return list(itertools.product(params1, params2))

and then map over the output of

create_product

Thanks for your response! It might be too brute-force since I don't need the product list itself. Is there a more memory-efficient way than to create and pass back the entire product list (params1 and params2 could have length of 1000+)? I'm pretty new to Prefect and not sure about memory management in tasks.

Anytime! It’s an interesting question; unfortunately I can’t think of a more memory efficient design at the moment - Prefect was designed to run in a distributed context, so all outputs are required to be serializable - this means that iterators aren’t supported as outputs at this time. I’ll give it some thought though and maybe there’s a more clever design lurking in here

So I can do something like?

@task
do_something = lambda p1, p2: one_thing(p1, p2)

prod = create_product(p1, p2) 
result = do_something.map(prod)