https://prefect.io logo
Title
d

Dee M

01/20/2021, 7:48 PM
Hi everyone! I have the below Prefect flow with ECSRun run configration. On registering the flow, I receive the below error. How can I override the task def parameters? Error
Parameter validation failed: Invalid type for parameter containerDefinitions[0].logConfiguration.options.awslogs-create-group, value: True, type: <class 'bool'>, valid types: <class 'str'>
z

Zanie

01/20/2021, 7:57 PM
Hi! Would you mind moving the code into this thread so it doesn’t fill up as much of the channel?
m

Mariia Kerimova

01/20/2021, 7:57 PM
Hello Dee! You can provide task definition or task_definition_path in ECSRun. The same way you passed cpu, memory and env. Refer to this.
d

Dee M

01/20/2021, 8:39 PM
Code
config = ECSRun(cpu=512, memory=1024,
        env={
            'FLOW_NAME': 'my-prefect'
        })

executor = LocalDaskExecutor()

result = S3Result(bucket=prefect_bucket)

store = Docker(
    registry_url="...",
    image_name="my_prefect",
    image_tag="my_prefect,
    base_image="...",
    python_dependencies=['boto3'],
    files={


    path="my_prefect.py",
    stored_as_script=True,
    ignore_healthchecks=True
)

with Flow("my-prefect",
    storage=store,
    run_config=config,
    result=result,
    executor=executor
) as flow:

    raw = get_data()
    df = transform_data(raw)
    save_data(df)
Thanks @Mariia Kerimova. Is there a way to override only the container definition (specifically log configuration) and not update the full task def?
@Zanie Done
m

Mariia Kerimova

01/20/2021, 8:52 PM
Right now there is only an option to provide the task definition/tf path or arn within ECSRun. It would be great if you will have a chance to can open an issue for adding configuration for container definitions.
d

Dee M

01/20/2021, 8:59 PM
Can this be accomplished using
run_task_kwargs
?
m

Mariia Kerimova

01/20/2021, 9:06 PM
I just skimmed aws documentation for run_task and didn't find where you can set logConfiguration
j

Jim Crist-Harif

01/20/2021, 9:46 PM
Hmmm, we don't set
logConfiguration
anywhere in the ECS agent - the error you're seeing seems to indicate that it's being set incorrectly, but we don't set it at all by default. Have you set it as a default on your agent?
d

Dee M

01/20/2021, 9:50 PM
@Jim Crist-Harif I did (in the registry_url in storage.Docker) but I definitely set the value to “true” not True as stated in the error...
j

Jim Crist-Harif

01/20/2021, 9:53 PM
I'm not sure I follow - how is the
registry_url
related to the
logConfiguration
of an ECS task definition?