https://prefect.io logo
e

E Li

02/02/2022, 4:08 PM
Hi, my task is mapped to some children tasks. if some of the children tasks fail in a run, is there a way to skip the successful ones and only re-run the failed? Can this be achieved by state handlers? thanks
k

Kevin Kho

02/02/2022, 4:13 PM
When you mean re-run, are you clicking the restart button in the UI? Or are you kicking off a new flow run and only want to re-run the failed?
a

Anna Geller

02/02/2022, 4:16 PM
Also, if you configure automatic retries on your tasks, this should retry failed child task runs automatically without restarting successful child task runs
e

E Li

02/02/2022, 4:22 PM
the later. i am thinking of the situation where there might be issues specific (like missing input files) to certain child tasks, so re-run is needed after manually fix that issue
k

Kevin Kho

02/02/2022, 4:26 PM
You need caching , which will take effect across flow runs. You need to use templating so each mapped task has a unique file
e

E Li

02/02/2022, 8:18 PM
so for @task(target="func_task_target.txt", checkpoint=True, result=LocalResult(dir="~/.prefect"), if there is no such func_task_target.txt in ~/.prefect, then prefect will create that txt file and write the return value of my task into it?
k

Kevin Kho

02/02/2022, 8:21 PM
Yes exactly but for mapped tasks, you want different file names for each task
e

E Li

02/02/2022, 8:23 PM
ok, for some reason, i don't see the txt being created in the dir
k

Kevin Kho

02/02/2022, 8:24 PM
Are you using
flow.run()
or an agent run?
e

E Li

02/02/2022, 8:24 PM
flow.run()
i passed an absolute path, like C:\\Users\\yliyunhu\\.prefect\\results
k

Kevin Kho

02/02/2022, 8:25 PM
It won’t persist for
flow.run()
. Maybe if you have an env variable
Copy code
PREFECT__FLOWS__CHECKPOINTING=true
But the agent run will persist it
e

E Li

02/02/2022, 8:34 PM
i see, i tried this flow.run_config=LocalRun(env="PREFECT__FLOWS__CHECKPOINTING": True}), is there another way i can try?
k

Kevin Kho

02/02/2022, 8:41 PM
flow.run()
does not respect the RunConfig so you really need an environment variable or to put it in the
config.toml
This environment variable is automatically set to true for runs against cloud or server
e

E Li

02/02/2022, 8:57 PM
ok that worked. how does it work with cache_validator and cache_for? say if i pass a different set of inputs, will it bypass the target and invalidate the cache?
k

Kevin Kho

02/02/2022, 9:08 PM
Targets and cache_for + cache validator are two independent caching mechanisms are you should only need one
cache_validator cannot invalidate the target. You need to delete it or use a filename that contains some rounded timestamp
e

E Li

02/02/2022, 9:33 PM
got it, thank you
hi, just to confirm, does this callable name generators only work with agent run? b/c with flow.run, i see in logs that the task function name is used as task_run_name
k

Kevin Kho

02/04/2022, 4:40 PM
Yes it’s with a backend run
e

E Li

02/04/2022, 4:44 PM
got it, thanks
4 Views