https://prefect.io
Join Slack
Heya! Hopefully this is an easy thing to do: I have a flow that calls bunch of subflows. Each subflo...
# ask-community
j
Heya! Hopefully this is an easy thing to do: I have a flow that calls bunch of subflows. Each subflow processes an "entity" e.g. product, discount etc. How can I give these subflows an appropriate name? Right now it is a dictionary generated pairs e.g.
Copy code
currently   => desired
---
smoky-moose => product
giga-locus  => discount
and so on.. This would be a massively positive change when it comes to monitoring the flows 🤔
✅ 1
The subflows as of right now are called in a loop in the main flow
Copy code
for entity in entities:
        <http://logger.info|logger.info>(f"Processing entity '{entity}'")
        process_entity(
            entity=entity,
        )
and the subflow is defined as
Copy code
@flow(
    cache_result_in_memory=False,
    task_runner=SequentialTaskRunner(),
)
def process_entity(
    entity,
) -> None:
    logger = get_run_logger()
    ...
j
hey, you can use 
@flow(flow_run_name=...)
when defining your subflows
It can be provided as a string template with the flow's parameters as variables or it can also be provided as a function that returns a string
e.x. for the string templating:
Copy code
from prefect import flow

@flow(flow_run_name="subflow1-{param1}")
def subflow1(param1):
    print("Hello from a subflow!")

@flow(log_prints=True)
def my_flow():
    print("Hello from a flow!")
    subflow1("my-very-first-param")


if __name__ == '__main__':
    my_flow()
🙌 1
j
Brilliant! I did not know you can inject the function args like that. Thank you @Jake Kaplan!
🙌 2
t
Now that is cool! I didn't know either!
4 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by