https://prefect.io
Join Slack
Hey, I’m in the process of switching to Prefect 2.x. I’m trying to figure out how to map over multip...
# ask-community
j
Hey, I’m in the process of switching to Prefect 2.x. I’m trying to figure out how to map over multiple arguments and generate child subflows. I realize that 
Task.map
can do something similar to generate tasks, but I’d prefer subflows for my project. Is this possible? If so, any docs or resources y’all can point me to? Thanks!
I’m aware of this link. Perhaps I’m misunderstanding subflows. Regardless, any help would be appreciated. https://docs.prefect.io/faq/#does-prefect-2-support-mapping
j
Can you show us what you have so far? The subflows can be simply another flow that you call within parent flow. You can call it like you would normally call another Python function. It just needs the 
@flow
decorator.
👍 2
j
so i understand how to generate a single subflow. but is there an equivalent 
.map
method that generates multiple subflows?
j
So, I’ve not used Prefect before 2.5 … I simply create methods and any method that has tasks that I need to call in another flow must also be a flow. Anything that doesn’t need to be a flow or is a small chunk… it is a task. So for me some of the work at the beginning of my flow were tasks … then I would usually have some sort of subflow or two as the next steps, and usually a couple of tasks before returning my final result if I need it stored.
I hope that helps…
Maybe one bit that might be helpful… the bits in a for loop body are usually subflows as they likely will need to call some tasks. Maybe that’s what you’re looking for with the “generate multiple subflows”?
j
fyi i modified one of the prefect tutorials and it looks like a simple 
for
loop does the job
🎉 1
Copy code
import requests
from prefect import flow

@flow
def call_api(url):
    response = requests.get(url)
    print(response.status_code)
    return response.json()

@flow
def api_flow(url_list: list[str]) -> list[dict]:
    for url in url_list:
        fact_json = call_api(url)
    return fact_json

url_list = [
    "<https://catfact.ninja/fact>",
    "<https://catfact.ninja/fact>"
]

json_list = api_flow(url_list)

print(json_list)
j
John, I hope that fully solves your issue!
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by