Is it wrong to pass list as a task parameter? In t...
# prefect-community
k
Is it wrong to pass list as a task parameter? In the following example the list items order is not preserved.
Copy code
from prefect import Flow, task


@task
def arrtest(arr):
    return ''.join(arr)


with Flow('ArrayTest') as flow:
    r = arrtest(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k'])

print(flow.run().result[r].result)
Expected result: abcdefghijk Actual result: ajkbcdefghi
j
Hi Kamil, thanks for the question. I'm looking into it for you now.
Although a bit surprising, this is expected behavior. Prefect turns the list and all of its elements into tasks/pseudo tasks and tasks run in an arbitrary order.
To solve it you can wrap the input in a
Constant
which prevents Prefect's auto-task mechanism from parsing it.
Copy code
list_input =prefect.tasks.core.constants.Constant([1,2,3])
Let us know if you have any other questions.
k
Thank you for quick answer, Jenny!
a
Hello you guys, I was wondering what are the motivation behind not preserving the order ? In my case I have images and labels, and if I'm modifying my images with a task I'm scared they will not match my labels anymore :s
@josh @nicholas Now that you ask if there is any other question 😛 I'm concern about this thread. I tested and it seems that it doesn't reorder if the list in part of the parameters of the flow. is there a precise rule for when the list is reordered ? 🙂 (after this one, I swear I stop asking questions :p)
j
We love questions here! You are correct in that the list will not reorder if it is passed in as a
Parameter
because Parameters are a special kind of task. What is happening here is a list is being defined directly in the context of the Flow and therefore is being converted into a
Constant
(which is also a special type of task).
upvote 1
👍 1
n
@alexandre kempf I think the takeaway here is that not defining parameters as constants means you can't guarantee that tasks will run (or in this case exist) in a particular order. This is because tasks can be run in a distributed environment where certain clusters will pick up certain tasks as they're available
👍 1
a
From what I understood, the list in that example is not a
Constant
, and this is why it can be reorder. Can a task (here
r
for example) be reorder if it goes through another task ? (Imagine
r
is a task with a list as a result, and you give it to another task) or task are also a special type ? If this question is not clear I can provide an example, don't hesitate to ask about it if needed 🙂
Basically, what I want to know is, if all my inputs for a flow are parameters, can I have reordered list at some point 🙂
j
Nope that list will not be reordered!
a
That is awesome ! Thank you again 😄
m
I think it might be instructive to compare the visualization of the constant list and non-constant list version. I think the constant-list version should have one input while the non-constant list version you should see many.
upvote 1