Though that works to set `PREFECT__FLOWS__CHECKPOINTING=true`, what is actually happening regarding ...

Josh Pitts

01/10/2021, 2:30 AM

Though that works to set

PREFECT__FLOWS__CHECKPOINTING=true

, what is actually happening regarding

target

checkpoint=True

, and

result=LocalTarget()

? or are those just ignored?

Jackson Maxfield Brown

01/10/2021, 3:16 AM

target

uses checkpointing, so if a result already exists at that filepath, the task will be skipped and the data will only be read when needed downstream. If you replace

target

with

location

it will always be overwritten each run.

Josh Pitts

01/10/2021, 3:21 AM

I tried replacing

target

with

location

but i am getting an error. I don’t see

location

as a part of the Task API (https://docs.prefect.io/api/latest/core/task.html#task-2)

Jackson Maxfield Brown

01/10/2021, 3:22 AM

woops sorry

Jackson Maxfield Brown

01/10/2021, 3:22 AM

its part of the

Result

API

Jackson Maxfield Brown

01/10/2021, 3:22 AM

https://docs.prefect.io/api/latest/engine/result.html#result

Jackson Maxfield Brown

01/10/2021, 3:22 AM

so you could drop

target

on the task but add

location

to the result params

Josh Pitts

01/10/2021, 3:26 AM

ok, great, yes, this worked great. I am past the toy examples in the docs but not quite ready for the advanced stuff, so the donut I am in is a bit confusing. Thanks for your help

3 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.