https://prefect.io logo
Title
m

Matthias

07/17/2020, 9:44 AM
Hi, I have the following situation: When I run this example class, it creates a persisted result stored on disk when I
flow.register()
and run via UI. When run via
flow.run()
it does not store anything. What do I have to do to not store anything? This keeps filling up storage on my server. It is just an example, it seems to me to be related to the mapping.
from prefect import Flow, task
from string import ascii_lowercase


@task(checkpoint=False)
def extract_things(c):
    return c

with Flow(
    "Do it",
) as flow:
    extract_things.map(list(ascii_lowercase))
j

josh

07/17/2020, 11:20 AM
Hi @Matthias in your example you are creating a Constant task in prefect by passing that list directly into the
.map
. Constants are special tasks in prefect with their own logic and if you want to control that behavior you should instead wrap it in a task declaration like this:
@task(checkpoint=False)
def get_list():
    return list(ascii_lowercase)

@task(checkpoint=False)
def extract_things(c):
    return c

with Flow(
    "Do it",
) as flow:
    l = get_list()
    extract_things.map(l)
m

Matthias

07/17/2020, 11:43 AM
I have a similar behaviour when a Task uses a
Parameter
. As Parameter is a also a special task, os there a way to disable checkpointing here as well, @josh?
j

josh

07/17/2020, 12:13 PM
Can you share a snippet of your flow? The Parameter should have a default of a PrefectResult which persists in the DB as opposed to a local file
m

Matthias

07/17/2020, 12:48 PM
Ah, it seems not to be related to the Paramter. I have to dig deeper into what is happening and will come back. Thanks for the hint already, that solved some of the persistencies I have 🙂 Actually a suggestion would be to have a central config item to no persist anything to disk.