https://prefect.io logo
Title
k

Kamil Okáč

11/26/2019, 9:49 AM
Is it wrong to pass list as a task parameter? In the following example the list items order is not preserved.
from prefect import Flow, task


@task
def arrtest(arr):
    return ''.join(arr)


with Flow('ArrayTest') as flow:
    r = arrtest(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k'])

print(flow.run().result[r].result)
Expected result: abcdefghijk Actual result: ajkbcdefghi
j

Jenny

11/26/2019, 1:57 PM
Hi Kamil, thanks for the question. I'm looking into it for you now.
Although a bit surprising, this is expected behavior. Prefect turns the list and all of its elements into tasks/pseudo tasks and tasks run in an arbitrary order.
To solve it you can wrap the input in a
Constant
which prevents Prefect's auto-task mechanism from parsing it.
list_input =prefect.tasks.core.constants.Constant([1,2,3])
Let us know if you have any other questions.
k

Kamil Okáč

11/26/2019, 3:02 PM
Thank you for quick answer, Jenny!
a

alexandre kempf

12/04/2019, 3:59 PM
Hello you guys, I was wondering what are the motivation behind not preserving the order ? In my case I have images and labels, and if I'm modifying my images with a task I'm scared they will not match my labels anymore :s
@josh @nicholas Now that you ask if there is any other question 😛 I'm concern about this thread. I tested and it seems that it doesn't reorder if the list in part of the parameters of the flow. is there a precise rule for when the list is reordered ? 🙂 (after this one, I swear I stop asking questions :p)
j

josh

12/04/2019, 4:37 PM
We love questions here! You are correct in that the list will not reorder if it is passed in as a
Parameter
because Parameters are a special kind of task. What is happening here is a list is being defined directly in the context of the Flow and therefore is being converted into a
Constant
(which is also a special type of task).
:upvote: 1
👍 1
n

nicholas

12/04/2019, 4:37 PM
@alexandre kempf I think the takeaway here is that not defining parameters as constants means you can't guarantee that tasks will run (or in this case exist) in a particular order. This is because tasks can be run in a distributed environment where certain clusters will pick up certain tasks as they're available
👍 1
a

alexandre kempf

12/04/2019, 4:44 PM
From what I understood, the list in that example is not a
Constant
, and this is why it can be reorder. Can a task (here
r
for example) be reorder if it goes through another task ? (Imagine
r
is a task with a list as a result, and you give it to another task) or task are also a special type ? If this question is not clear I can provide an example, don't hesitate to ask about it if needed 🙂
Basically, what I want to know is, if all my inputs for a flow are parameters, can I have reordered list at some point 🙂
j

josh

12/04/2019, 4:45 PM
Nope that list will not be reordered!
a

alexandre kempf

12/04/2019, 4:46 PM
That is awesome ! Thank you again 😄
m

Mark Koob

12/04/2019, 9:02 PM
I think it might be instructive to compare the visualization of the constant list and non-constant list version. I think the constant-list version should have one input while the non-constant list version you should see many.
:upvote: 1