Channels
#prefect-community
Title
k
Kyle Pierce
10/02/2020, 6:46 PMI have a task that take two parameters and outputs a list. Whenever i try to register it it gives me this error
TypeError: 'FunctionTask' object is not iterable👀 1
Copy code
with Flow("Load Daily Tables", schedule) as flow:
start_date = Parameter('start_date', default='')
end_date = Parameter('end_date', default='')
dates = daterange(start_date, end_date)
# dates = ['2020-09-29', '2020-09-30']
@task(log_stdout=True)
def daterange(start_date, end_date):
print(start_date, end_date)
list_of_dates = []
# Keep the parameters short you can leave end_date blank and it will default to today.
if end_date == '':
eastern = timezone('US/Eastern')
single_date = datetime.now(eastern)
end_date = single_date
# If start_date is blank we are just going to pull yesterday
if start_date == '':
eastern = timezone('US/Eastern')
start_date = datetime.now(eastern)
elif start_date != '':
start_date = datetime.strptime(start_date, '%Y-%m-%d')
# Make start_date eastern
start_date = start_date.replace(tzinfo=eastern)
# Otherwise we are going to get a list of dates.
for n in range(int((end_date - start_date).days)):
single_date = (end_date - timedelta(n)).strftime("%Y-%m-%d")
list_of_dates.append(single_date)
print(list_of_dates)
return list_of_datesd
Dylan
10/02/2020, 7:04 PM
Hi @Kyle Pierce!
Copy code
from prefect import task, Flow, Parameter
@task(log_stdout=True)
def daterange(start_date, end_date):
print(start_date, end_date)
list_of_dates = []
# Keep the parameters short you can leave end_date blank and it will default to today.
if end_date == "":
eastern = timezone("US/Eastern")
single_date = datetime.now(eastern)
end_date = single_date
# If start_date is blank we are just going to pull yesterday
if start_date == "":
eastern = timezone("US/Eastern")
start_date = datetime.now(eastern)
elif start_date != "":
start_date = datetime.strptime(start_date, "%Y-%m-%d")
# Make start_date eastern
start_date = start_date.replace(tzinfo=eastern)
# Otherwise we are going to get a list of dates.
for n in range(int((end_date - start_date).days)):
single_date = (end_date - timedelta(n)).strftime("%Y-%m-%d")
list_of_dates.append(single_date)
print(list_of_dates)
return list_of_dates
with Flow("Load Daily Tables") as flow:
start_date = Parameter("start_date", default="")
end_date = Parameter("end_date", default="")
dates = daterange(start_date, end_date)
# dates = ['2020-09-29', '2020-09-30']This registered for me just fine
(I removed the
schedulek
Kyle Pierce
10/02/2020, 7:05 PMsorry left out one extra part @Dylan
Copy code
with Flow("Load Daily Tables") as flow:
start_date = Parameter("start_date", default="")
end_date = Parameter("end_date", default="")
dates = daterange(start_date, end_date)
# dates = ['2020-09-29', '2020-09-30']
table_list = ['flows', 'sms_replies']
# Two lists. dates and tables
# [date1, table1], [date1, table2], [date2, table1], [date2, table2]
combos = list(product(table_list, dates))d
Dylan
10/02/2020, 7:06 PM
what’s
productBut, generally speaking
There’s a different between flow build time and flow execution time. Task evaluation is deferred until execution time
So if you’re directly trying to turn something into a
list Copy code
(prefect) dylanhughes@Dylans-MacBook-Pro-Prefect ~/Desktop> ipython -i test.py
Python 3.7.7 (default, Mar 26 2020, 10:32:53)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
~/Desktop/test.py in <module>
35 # Two lists. dates and tables
36 # [date1, table1], [date1, table2], [date2, table1], [date2, table2]
---> 37 combos = list(product(table_list, dates))
NameError: name 'product' is not definedError for reference
A more eloquent explanation ☝️
k
Kyle Pierce
10/02/2020, 7:20 PM@Dylan thanks for you help i must of had a typo somewhere. It looks like you guys added a datetime parameter so i need to update to the newest version. Again thanks for taking a look
👍 1
@Dylan one other quick question. Can you name mapped tasks?
d
Dylan
10/02/2020, 7:34 PM
That’s currently in the works
👍 1
k
Kyle Pierce
10/02/2020, 7:34 PMVery cool thank you!
👍 1