Hi there I have a module that users `loguru` <https github c Prefect Community #ask-community

Hi there. I have a module that users `loguru` <htt...

Pedro Machado

12/14/2021, 3:42 PM

Hi there. I have a module that users

loguru

https://github.com/Delgan/loguru It's used inside of a task. When I run the flow locally, I see the loguru output but when it runs remotely (ECS), I don't see the logs in Prefect Cloud. How could I get it to write to the Prefect logs? Also, how should I set up my ECS flow to get it to write the container logs to cloudwatch? Thanks!

Kevin Kho

12/14/2021, 3:47 PM

Hey @Pedro Machado, add the extra loggers here to Prefect for the logs to show up. This will not work for Dask executor.

Kevin Kho

12/14/2021, 3:47 PM

For logs to cloud watch, you need to configure it like this:

Copy code

task_definition = yaml.safe_load(
      """
      cpu: 1024
      memory: 2048
      containerDefinitions:
      - name: flow
        logConfiguration:
          logDriver: awslogs
          options:
            awslogs-group: /ecs/prefect-agent
            awslogs-region: us-east-1
            awslogs-stream-prefix: ecs
            awslogs-create-group: 'true'
      """

Kevin Kho

12/14/2021, 3:51 PM

There is a loguru thread somewhere up here I think in case you are curious

Kevin Kho

12/14/2021, 3:51 PM

Ah I guess it’s gone

Pedro Machado

12/14/2021, 5:17 PM

Thanks Kevin. I think I a missing something. I tried this env var on the ECSRun and got the snowflake connector to log but not my custom library. I am thinking I need to do something on the loguru side. Any ideas?

Copy code

PREFECT__LOGGING__EXTRA_LOGGERS="['snowflake.connector', 'boto3', 'custom_lib']"

Kevin Kho

12/14/2021, 5:19 PM

May try

Copy code

@task(log_stdout=True)

if the logs go to stdout? Or maybe add loguru?

41 Views

Open in Slack

Previous Next