Channels
pacc-may-31-2023
prefect-ai
pacc-clearcover-june-12-2023
marvin-in-the-wild
data-ecosystem
geo-israel
pacc-june-14-2023
geo-japan
prefect-cloud
ppcc-may-16-2023
prefect-azure
prefect-docker
prefect-recipes
gratitude
geo-nyc
geo-bay-area
geo-boston
geo-london
geo-dc
geo-chicago
geo-berlin
geo-texas
geo-seattle
geo-colorado
prefect-community
data-tricks-and-tips
prefect-aws
prefect-gcp
introductions
find-a-prefect-job
prefect-dbt
random
events
ask-marvin
show-us-what-you-got
prefect-getting-started
prefect-integrations
prefect-contributors
best-practices-coordination-plane
announcements
prefect-server
prefect-ui
prefect-kubernetes
Title
j
John Ramey
10/18/2022, 2:03 AMHey, I’m in the process of switching to Prefect 2.x. I’m trying to figure out how to map over multiple arguments and generate child subflows. I realize that
Task.mapI’m aware of this link. Perhaps I’m misunderstanding subflows. Regardless, any help would be appreciated. https://docs.prefect.io/faq/#does-prefect-2-support-mapping
j
Jarvis Stubblefield
10/18/2022, 8:04 AMCan you show us what you have so far? The subflows can be simply another flow that you call within parent flow. You can call it like you would normally call another Python function. It just needs the
@flow👍 2
j
John Ramey
10/19/2022, 1:58 AMso i understand how to generate a single subflow. but is there an equivalent
.mapj
Jarvis Stubblefield
10/19/2022, 5:47 AMSo, I’ve not used Prefect before 2.5 … I simply create methods and any method that has tasks that I need to call in another flow must also be a flow. Anything that doesn’t need to be a flow or is a small chunk… it is a task. So for me some of the work at the beginning of my flow were tasks … then I would usually have some sort of subflow or two as the next steps, and usually a couple of tasks before returning my final result if I need it stored.
I hope that helps…
Maybe one bit that might be helpful… the bits in a for loop body are usually subflows as they likely will need to call some tasks. Maybe that’s what you’re looking for with the “generate multiple subflows”?
j
John Ramey
10/19/2022, 9:24 AMfyi i modified one of the prefect tutorials and it looks like a simple
for🎉 1
import requests
from prefect import flow
@flow
def call_api(url):
response = requests.get(url)
print(response.status_code)
return response.json()
@flow
def api_flow(url_list: list[str]) -> list[dict]:
for url in url_list:
fact_json = call_api(url)
return fact_json
url_list = [
"<https://catfact.ninja/fact>",
"<https://catfact.ninja/fact>"
]
json_list = api_flow(url_list)
print(json_list)j
Jarvis Stubblefield
10/19/2022, 7:19 PMJohn, I hope that fully solves your issue!