Anish Chhaparwal

    Anish Chhaparwal

    1 year ago
    hey i have a couple of question: is it possible to use cache_for in a ShellTask? How can i pass a parameter to command (f string) of a ShellTask. Eg:
    git_clone=ShellTask(log_stderr=True)
    with Flow ('QETL') as flow:
    git_url = Parameter("git_url",
                      
    default="<https://github.com/ieee8023/covid-chestxray-dataset>")
    git_clone(command="git clone {url} {target}".format(url=git_url)
    flow.run()
    Michael Adkins

    Michael Adkins

    1 year ago
    Parameters are not resolved until flow runtime so you can’t format the string there. You’d want to have a task that generates the command you want from the parameter e.g.
    make_git_clone_command(git_url)
    and pass that to
    ShellTask
    And yes
    cache_for
    should work for any subclass of
    Task