So, we run our dbt stuff via Prefect. We wanted to be able to see the compiled SQL queries, and found turning them into Flow Artifacts was the easiest way to do it! Shocked by how painless it was. Here's a few functions:

@task(trigger=all_finished)
def get_filepaths(dbt_path: str = "/root/dbt-repo/target") -> t.List[pathlib.PosixPath]:
    return list(pathlib.Path(f"{dbt_path}/compiled").rglob("*.sql"))


def make_query_name(path: pathlib.PosixPath) -> str:
    query_name = str(path).split("/compiled")[1]
    return f"# {query_name}"


def make_sql_markdown(sql: str) -> str:
    return f"

sql\n{sql}\n```" @task(trigger=all_finished) def publish_artifact(filepaths: t.List[pathlib.PosixPath]) -> None: titles_and_queries = [ "\n".join((make_query_name(path), make_sql_markdown(path.read_text()))) for path in filepaths ] all_merged = "\n\n".join(titles_and_queries) create_markdown(all_merged)```

This is very cool

This is awesome

This is REALLY awesome 😍