m

    Matt Alhonte

    11 months ago
    So I'm refactoring a Flow into a Flow of Flows. It runs a bunch of Papermill notebooks. They have to be run in a certain order - but in some cases, a dependency isn't relevant, so we skip it. This currently takes the form of a bunch of Tasks along the lines of
    if notebook_name in list_of_notebooks:
       run_notebook(notebook_name)
    else:
        print("skipped")
    How do I do that with the
    StartFlowRun
    task? Should I subclass it and replace the existing
    .run()
    method?
    Kevin Kho

    Kevin Kho

    11 months ago
    Subclassing the run makes sense here. Then you can just trigger the
    StartFlowRun
    call if it fulfills the condition.
    m

    Matt Alhonte

    11 months ago
    Cool, thanks! So I guess I'll move the current
    .run()
    fn to a different name (let's say
    .execute_flow
    , and then have the new
    .run()
    as something like
    if notebook_name in list_of_notebooks: self.execute_flow()
    ?
    Kevin Kho

    Kevin Kho

    11 months ago
    You can override the run and use super right? The DBTShellTask is a good example of this as it just overrides the ShellTask then calls
    super.run()
    to call the ShellTask
    run()