Thread
#prefect-community
    a

    Adam Roderick

    9 months ago
    Hi, does anyone have recommendations or best practices that they follow around dependencies? My specific example has to do with extraction/load vs transformation steps. It is convenient to put it all in one Flow so that the transformations start after the extraction/loads are complete. But our Flows have become quite monolithic, and we want to be able to test various sections (i.e. extract/load) separately from other steps (i.e. transformations). Any recommendations here?
    Anna Geller

    Anna Geller

    9 months ago
    @Adam Roderick you could explore the flow of flows pattern. This allows you to run the extract-load jobs separately from transformations when needed, while at the same time allowing to call all child flows in a parent flow that can run on schedule. We wrote about this in this blog post.