Hi! Why does this example lead to the creation of ...
# prefect-community
m
Hi! Why does this example lead to the creation of persisted results in the results folder? Thanks 🙂
Copy code
@task(checkpoint=False)
def extract_things():

    bar = [1, 2, 3, 4, 5, 6, 7, 8, 9]
    foo = ['a', 'b', 'c']

    data = {
        'bar': bar,
        'foo': foo
    }

    return data


@task(checkpoint=False)
def do_things(data):

    for x in data:
        print(x)

    return None

with Flow(
    "Do it",
) as flow:
    things = extract_things()
    do_things(things['foo'])
    do_things(things['bar'])
👀 1
n
Hi @Matthias - I can't seem to reproduce this; I've pinged some of the other Prefect team members to see if there's something I'm missing.
m
Some Background: I am registering the flow and running it from the UI, when running it with flow.run() it does not do that
n
Ah that's very helpful, thank you.
m
It creates two result-files, one for each key in the data dict in extract_things().
n
You'r right @Matthias - I'll open an issue for this, I think it's a bug.
m
Alright, interesting that no one else stumbled across this yet 😄 Is there an easy way to remove persisted data after a flow run? I have this happening quite a lot and after a couple of days the data fill up the servers hard disk
n
Ahh @Matthias, the issue is that you're accessing
things['foo']
and
things['bar']
directly when passing the result into your downstream tasks, which is turning them into a
Constant
task. You should instead access those props by mapping over them or accessing the keys directly in your tasks.
But I think this could be more intuitive so I'll open an issue to throw a warning
m
Oh, interesting! That helps a lot 🙂 I think actually there should be a way to prevent Constants from persisting data, does that make sense?
n
Definitely @Matthias - I've opened a ticket to discuss it here (I used your example)... feel free to chime in!
m
Thanks 🙂 And thank you for helping out. this was a big headache for me today 😄
👍 2