t

    tkanas

    2 years ago
    For a given flow, if a task fails I am wanting to write the state to my local machine, and be able to load it back up later. I am looking into state handlers and persisting output using
    LocalResult
    but I'm wondering if there are Prefect features that are particularly appropriate for this use case.
    Dylan

    Dylan

    2 years ago
    Hey @tkanas! When you say “state” here, can you elaborate a bit? Do you want to write the result of previous tasks, the flow context, the state object for that task run?
    t

    tkanas

    2 years ago
    That's something we are still thinking about. Basically the state of the flow just before failure, so that a user could possibly load it back up and inspect it later.
    Dylan

    Dylan

    2 years ago
    Hmmm, but this is in response to failure?
    So you want to catch all exceptions, write something down, and then fail the task run?
    t

    tkanas

    2 years ago
    I think so. I'm still thinking about what the "something" is and what the best way is of "writing it down". So far I have been looking at using a
    state_handler
    (that would check
    state.is_failed()
    and then dispatch to the thing that writes the state) in conjunction with a persisting method as described in the link you provided above. I just haven't figured out the best way of doing it.
    Dylan

    Dylan

    2 years ago
    So I think the key point here is that you want to know the state “before” the failure
    I’m not sure a state handler will help with that, since they fire on state change
    However, have you tried setting up Prefect Server or Prefect Cloud? If you have an orchestration layer you would be able to see the state of the failed flow along with logs
    Or do you want to “load” a certain state of the world into a python interpreter locally?
    Another way of asking my question: are you having trouble identifying an issue inside a task, or testing a solution?
    t

    tkanas

    2 years ago
    Saving and loading a state of the world sounds like it might fit the bill. Preferably the state right before failure. If that's not possible, then maybe the state right after failure?
    How would you save and load a "state of the world"?
    Dylan

    Dylan

    2 years ago
    Locally, I’m not sure we have a way to do this
    But if you’re using Prefect Server/Cloud, you can manipulate task run states and flow run states with the graphql API
    that would allow you to watch a flow run in a given state of the world
    t

    tkanas

    2 years ago
    OK, I will look into that. Thanks!
    Dylan

    Dylan

    2 years ago
    Anytime!