Charles Leung
08/17/2023, 4:17 PMMoe
08/17/2023, 4:32 PMCharles Leung
08/17/2023, 6:01 PMMoe
08/17/2023, 7:46 PMfile:
level: 0
class: logging.handlers.TimedRotatingFileHandler
filename: /var/log/prefect.log
when: 'D'
interval: 1
backupCount: 7
formatter: standard
Charles Leung
08/17/2023, 8:06 PMhandlers:
file:
level: 0
# The handlers we define here will output all logs they receieve by default
# but we include the `level` so it can be overridden by environment
class: logging.handlers.RotatingFileHandler
# Update filename to the location you want to store Prefect's log files.
# This location must be writable by the user account that runs Prefect.
filename: C:\Users\qtask\prefect.log
# maximum 10MB per log file
maxBytes: 10485760
# keep the last 5 log files in addition to the current log file
backupCount: 5
formatter: standard
Moe
08/17/2023, 8:11 PMprefect server
is the handler responsible for creating logs in the UI, right?loggers:
prefect:
level: "${PREFECT_LOGGING_LEVEL}"
prefect.extra:
level: "${PREFECT_LOGGING_LEVEL}"
handlers: [prefect server, file]
prefect.flow_runs:
level: NOTSET
handlers: [prefect server, file]
prefect.task_runs:
level: NOTSET
handlers: [prefect server, file]
prefect.infrastructure:
level: "${PREFECT_LOGGING_LEVEL}"
handlers: [prefect server, file]
I have the loggers set this way, but it isn't generating any logs.Charles Leung
08/18/2023, 6:16 PMJake Kaplan
08/18/2023, 7:22 PMMoe
08/18/2023, 7:22 PMJake Kaplan
08/18/2023, 7:23 PMfrom prefect import flow, task
from prefect.logging import get_run_logger
from logging import FileHandler
from prefect.runtime import flow_run
def _get_logger():
handler = FileHandler(filename=f"{flow_run.name}.log")
logger = get_run_logger()
# prefect uses a log adapter, so add to the underlying logger
original_logger = logger.logger
handler.setFormatter(original_logger.root.handlers[0].formatter)
original_logger.addHandler(handler)
return logger
@task
def my_task():
logger = _get_logger()
<http://logger.info|logger.info>("Hello from my task!")
@flow
def my_flow():
logger = _get_logger()
<http://logger.info|logger.info>("Hello from my Flow!")
my_task()
my_flow()
Moe
08/18/2023, 8:06 PMJake Kaplan
08/18/2023, 8:10 PMCharles Leung
08/22/2023, 11:06 PMTraceback (most recent call last):
File "//v-tm-qeq-05/PrefectScripts/cleung/Script/logging.py", line 1, in <module>
from prefect import flow, task, get_run_logger
File "\\v-tm-qeq-05\Prefect_Env\lib\site-packages\prefect\__init__.py", line 37, in <module>
from prefect.states import State
File "\\v-tm-qeq-05\Prefect_Env\lib\site-packages\prefect\states.py", line 9, in <module>
import anyio
File "\\v-tm-qeq-05\Prefect_Env\lib\site-packages\anyio\__init__.py", line 103, in <module>
from ._core._fileio import AsyncFile, Path, open_file, wrap_file
File "\\v-tm-qeq-05\Prefect_Env\lib\site-packages\anyio\_core\_fileio.py", line 24, in <module>
from .. import to_thread
File "\\v-tm-qeq-05\Prefect_Env\lib\site-packages\anyio\to_thread.py", line 7, in <module>
from .abc import CapacityLimiter
File "\\v-tm-qeq-05\Prefect_Env\lib\site-packages\anyio\abc\__init__.py", line 83, in <module>
from ..from_thread import BlockingPortal
File "\\v-tm-qeq-05\Prefect_Env\lib\site-packages\anyio\from_thread.py", line 4, in <module>
from asyncio import iscoroutine
File "\\v-tm-qeq-05\Prefect_Env\lib\asyncio\__init__.py", line 8, in <module>
from .base_events import *
File "\\v-tm-qeq-05\Prefect_Env\lib\asyncio\base_events.py", line 18, in <module>
import concurrent.futures
File "\\v-tm-qeq-05\Prefect_Env\lib\concurrent\futures\__init__.py", line 8, in <module>
from concurrent.futures._base import (FIRST_COMPLETED,
File "\\v-tm-qeq-05\Prefect_Env\lib\concurrent\futures\_base.py", line 7, in <module>
import logging
File "\\v-tm-qeq-05\PrefectScripts\cleung\Script\logging.py", line 1, in <module>
from prefect import flow, task, get_run_logger
ImportError: cannot import name 'flow' from partially initialized module 'prefect' (most likely due to a circular import) (\\v-tm-qeq-05\Prefect_Env\lib\site-packages\prefect\__init__.py)
Jake Kaplan
08/23/2023, 2:06 PMlogging.py
something else, i've seen that lead to circular import issues. Aside from that it is hard to say, it's very case by case for your own code 🙂Charles Leung
08/23/2023, 5:23 PM