Hey, I’m in the process of switching to Prefect 2....
# ask-communityj
John Ramey
10/18/2022, 2:03 AMHey, I’m in the process of switching to Prefect 2.x. I’m trying to figure out how to map over multiple arguments and generate child subflows. I realize that
Task.mapJohn Ramey
10/18/2022, 2:04 AMI’m aware of this link. Perhaps I’m misunderstanding subflows. Regardless, any help would be appreciated. https://docs.prefect.io/faq/#does-prefect-2-support-mapping
j
Jarvis Stubblefield
10/18/2022, 8:04 AMCan you show us what you have so far? The subflows can be simply another flow that you call within parent flow. You can call it like you would normally call another Python function. It just needs the
@flow👍 2
j
John Ramey
10/19/2022, 1:58 AMso i understand how to generate a single subflow. but is there an equivalent
.mapj
Jarvis Stubblefield
10/19/2022, 5:47 AMSo, I’ve not used Prefect before 2.5 … I simply create methods and any method that has tasks that I need to call in another flow must also be a flow. Anything that doesn’t need to be a flow or is a small chunk… it is a task. So for me some of the work at the beginning of my flow were tasks … then I would usually have some sort of subflow or two as the next steps, and usually a couple of tasks before returning my final result if I need it stored.
Jarvis Stubblefield
10/19/2022, 5:48 AMI hope that helps…
Jarvis Stubblefield
10/19/2022, 5:49 AMMaybe one bit that might be helpful… the bits in a for loop body are usually subflows as they likely will need to call some tasks. Maybe that’s what you’re looking for with the “generate multiple subflows”?
j
John Ramey
10/19/2022, 9:24 AMfyi i modified one of the prefect tutorials and it looks like a simple
for🎉 1
John Ramey
10/19/2022, 9:24 AM Copy code
import requests
from prefect import flow
@flow
def call_api(url):
response = requests.get(url)
print(response.status_code)
return response.json()
@flow
def api_flow(url_list: list[str]) -> list[dict]:
for url in url_list:
fact_json = call_api(url)
return fact_json
url_list = [
"<https://catfact.ninja/fact>",
"<https://catfact.ninja/fact>"
]
json_list = api_flow(url_list)
print(json_list)j
Jarvis Stubblefield
10/19/2022, 7:19 PMJohn, I hope that fully solves your issue!
4 Views