Hi all, I am trying to run flow using ECS Fargate (Following the wonderful guide at https://towardsdatascience.com/how-to-cut-your-aws-ecs-costs-with-fargate-spot-and-prefect-1a1ba5d2e2df#5b72) However, I am running into an issue with python versioning, as I am getting some syntax errors for f-string using f"{var=}" formatting that was introduced in python 3.8 - Although in the task definition it seems that I AM using python 3.8 as I'm using the prefecthq/prefect:latest-python3.8 image - What am I missing?

@Ido Slonimsky Where do you get this syntax error - in a flow? If so, could you share this flow definition?

Yes, I am seeing it in the prefect UI, the first line in the flow is:

@task()
def extract_data(source):
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>(f"{source=}")

With source being a parameter:

with Flow(FLOW_NAME, storage=STORAGE, run_config=RUN_CONFIG) as flow:
    source_name = Parameter("source")
    data = extract_data(source_name)

I see. I think the issue is that logging is different than print. The syntax you mentioned only works in print statements. Changing your task as follows should fix it:

@task(log_stdout=True)
def extract_data(source):
    print(f"{source=}")

Oh, got it, I'll change accordingly then, thanks!

btw @Ido Slonimsky, it’s always useful to set a default value for Parameter:

source_name = Parameter("source", default="your_default_source")

This way, you can schedule your flow to run with this default value and you can optionally overwrite it at runtime when needed.

Oh, that's neat! Awesome, thanks!

@Ido Slonimsky I couldn’t reproduce the error. Can you try this simple flow?

from prefect import task, Flow, Parameter


@task(log_stdout=True)
def hello_world(name):
    print(f"{name=}")


with Flow("mini.example") as flow:
    name = Parameter("name", default="world")
    hw = hello_world(name)

if __name__ == '__main__':
    flow.run()

It should give the output:

[2021-12-13 11:23:57+0100] INFO - prefect.TaskRunner | Task 'hello_world': Starting task run...
[2021-12-13 11:23:57+0100] INFO - prefect.TaskRunner | name='world'

Yeah, this runs with no problem

[2021-12-13 12:26:16+0200] INFO - prefect.FlowRunner | Beginning Flow run for 'mini.example'
[2021-12-13 12:26:16+0200] INFO - prefect.TaskRunner | Task 'name': Starting task run...
[2021-12-13 12:26:16+0200] INFO - prefect.TaskRunner | Task 'name': Finished task run for task with final state: 'Success'
[2021-12-13 12:26:16+0200] INFO - prefect.TaskRunner | Task 'hello_world': Starting task run...
[2021-12-13 12:26:16+0200] INFO - prefect.TaskRunner | name='world'
[2021-12-13 12:26:16+0200] INFO - prefect.TaskRunner | Task 'hello_world': Finished task run for task with final state: 'Success'
[2021-12-13 12:26:16+0200] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded

The flow I was trying to run also works correctly when using flow.run() - The problem occurs when I try to run it on the agents in the ECS Fargate

if this doesn’t work, I would suggest perhaps getting rid of this syntax? Honestly, if you hadn’t told me, I wouldn’t know about this syntax 😅 But the syntax below is what everyone understands:

@task(log_stdout=True)
def hello_world(name):
    print(f"name={name}")

Yeah I guess I'll get rid of it if I can't get it to work haha, I thought maybe there's a quick fix I was just missing but probably not worth spending too much time on it

@Ido Slonimsky to verify which Python version your ECS flow is using, you could try adding that to your task:

import platform
print(platform.python_version())

@Ido Slonimsky from the error message it looks like you were using “(source=)” instead of “{source=}” - we use this f-string syntax in our flows without any issues

@Marwan Sarieddine As shown in the code snippet, I was using curly braces, probably the display changes them

stange 🤷