Marwan Sarieddine
06/03/2020, 6:33 PMflow.run()
but flow.register()
flags an error
mainly the healthcheck throws this error - which I see referenced in the documentation … I am using prefect v0.11.4
ValueError: Mapped tasks with custom result locations must include
{filename} as a template in their location.
Please see the code below
import os
from prefect import task, Flow
from prefect.engine.results import S3Result
@task
def gen_list():
return [x for x in range(10)]
@task
def add(x, y):
return x + y
@task
def multiply(x, y):
return x * y
result = S3Result(
bucket=os.environ["AWS_BUCKET"],
location='prefect-testing/{task_name}/{map_index}.prefect'
)
with Flow('Test Flow', result=result) as flow:
x = gen_list()
y = gen_list()
added = add.map(x, y)
multiplied = multiply.map(added, added)
# flow runs fine if running locally
flow.run()
"""
flow fails to register
ValueError: Mapped tasks with custom result locations must include
{filename} as a template in their location.
"""
flow.register('Test Project')
nicholas
06/03/2020, 6:45 PM{filename}
in the location template for your result. So in this case you'd have something like:
# ...imports and tasks
result = S3Result(
bucket=os.environ["AWS_BUCKET"],
location="prefect-testing/{task_name}/{file_name}_{map_index}.prefect"
)
# ...
Marwan Sarieddine
06/03/2020, 6:46 PM{filename}
- but what I don’t understand is why {filename}
is necessary if flow.run() locally doesn’t throw errors and works fine ?nicholas
06/03/2020, 6:51 PMMarwan Sarieddine
06/03/2020, 6:55 PMfilename
attribute for local runs”nicholas
06/03/2020, 6:57 PMMarwan Sarieddine
06/03/2020, 6:57 PM