https://prefect.io logo
Title
k

karteekaddanki

07/21/2020, 10:27 PM
Hey guys, I am trying to use result targets to cache some of my results as suggested in https://docs.prefect.io/core/idioms/targets.html. However a lot of my heavyweight tasks are run in
C++
and are invoked via Python. How can I use targets effectively in this case? I've tried to
return None
but as expected, this causes my task to run always. I've worked around this issue by creating a target a layer of indirection by returning a path to a file that contains the filename of the actual file generated by my
C++
program (similar to empty targets in make, the presence of this file is indicates that the task is run and the contents of this file point to the location of the task output). It would be nice if I can avoid this indirection and directly be able to return a
Result
object that doesn't necessarily correspond to a serialized python object. All downstream processes that consume these results treat them as locations. In other words, I am looking for a behavior identical to Luigi. Thanks in advance.
z

Zachary Hughes

07/21/2020, 11:27 PM
Hi @karteekaddanki, Could I persuade you to open a Github issue explaining your use case? It's an interesting one, and having it in Github will let us socialize and track it.
k

karteekaddanki

07/22/2020, 6:55 AM