Channels
#prefect-community
Title
j
jeff n
12/30/2020, 12:41 AMI am struggling a bit with a parameter. It is being passed into my task at run time still a parameter. When I try to pass it to something expecting a string it doesn’t evaluate to the value. What am I missing on how to use parameters as normal values at run time.
j
Jenny
12/30/2020, 12:47 AMHi @jeff n - can you give a bit more information about how you're running your flow and passing your parameter?
j
jeff n
12/30/2020, 12:51 AM Copy code
def upload_to_s3(bucket_name, output_file):
"""
Using the following format to put the key
[system]/[topic]/[year]/[month]/[day]/[filename]
:param bucket_name:
:param output_file:
:return:
"""
s3 = boto3.resource('s3')
bucket = s3.Bucket(bucket_name)
objects = [obj for obj in bucket.objects.all()]
return objectsspecifically:
bucket = s3.Bucket(bucket_name) Copy code
with Flow("Event data pull") as flow:
event_date = Parameter("event_date", datetime(2020, 10, 1))
output_file = Parameter("output_file", "temp.csv")
bucket = Parameter("bucket", "lc-dateng-exports-dev")
events = pull_events_by_day(event_date)
write_to_csv = write_to_csv(output_file, events)
upload = upload_to_s3(bucket, output_file)
state = flow.run()That is the flow
j
Jenny
12/30/2020, 1:05 AMAre you adding an @task decorator to your upload_to_s3 function? Without that it'll run as part of the flow context instead of getting added as a task.
j
jeff n
12/30/2020, 4:23 AMWow way to see that right away. Thank you. I don’t know how I missed that but I would have never found it since it was looking at me in the face. So very appreciated.
👍 1