https://prefect.io logo
Join Slack
Powered by
I’ve attempted the solution in <https://prefect-co...
# prefect-community
j
I’ve attempted the solution in https://prefect-community.slack.com/archives/CL09KU1K7/p1661780834915459 However the regular Python logger will now raise:
Copy code
prefect.exceptions.MissingContextError: Logger 'dystematic.pdbt.ftp' attempted to send logs to Orion without a flow run id. The Orion log handler can only send logs within flow run contexts unless the flow run id is manually provided.
This isn’t covered in the post nor the docs. The reason I want to use the regular Python logs is because I cannot use the Prefect one when launching from the console as I get:
Copy code
RuntimeError: There is no active flow or task run context.
✅ 1
👀 1
b
Hello Jose, to avoid the RuntimeError you describe above, you need to set your logger and add your logs exclusively within tasks, flows, and subflows using 
get_run_logger
. This article may be helpful for you.
j
The article (and the others I have been pointed to) did not solve my issue. I know I need a flow run to get the logger and that is part of the issue. I depend on libraries that use the regular logging system and I need them to work. I cannot change them to receive a custom logger, neither I want to write my system around passing the logger to every single function that needs logging. There is something clearly failing in the communication between me and the Prefect engineers, so I will try to make this as clear as I can: • I have an FTP client which does some logging (info on the download status, warnings on message failures, and debug when necessary). ◦ This client has nothing to do with Prefect, thus, it should not use the Prefect logger, neither it should require me to pass it. • If I do not set the 
PREFECT_LOGGING_EXTRA_LOGGERS
variable (described in the blog post but not [explicitly] in the documentation), the flow runs fine but does not output logs • If I do set the 
PREFECT_LOGGING_EXTRA_LOGGERS
variable to the package(s) I want to log, I will: ◦ Get the logs (as expected) ◦ A 
UserWarning
:
Copy code
/Users/<REDACTED>/ftp.py:83: UserWarning: Logger 'dystematic.pdbt.ftp' attempted to send logs to Orion without a flow run id. The Orion log handler can only send logs within flow run contexts unless the flow run id is manually provided.
◦ On download of the second file, an error, which effectively stops the flow from progressing:
Copy code
prefect.exceptions.MissingContextError: Logger 'dystematic.pdbt.ftp' attempted to send logs to Orion without a flow run id. The Orion log handler can only send logs within flow run contexts unless the flow run id is manually provided.
However: • If I set the variable to 
PREFECT_LOGGING_EXTRA_LOGGERS = snowflake
, Snowflake’s logs work, and I doubt, their connector is importing Prefect’s 
get_run_logger
I’ve read: • https://discourse.prefect.io/t/how-to-use-logging-in-prefect-a-tutorial-by-andrew-brookins/197/3 • https://www.prefect.io/guide/blog/logs-the-prefect-way/ • https://discourse.prefect.io/t/can-i-define-the-logger-globally/114 • https://orion-docs.prefect.io/concepts/logs/ Furthermore, it’s the second time I get sent an article and my issue is marked as “solved” (or whatever ✅ means) without any reply from me — I still need time to read, evaluate and test the solution offered in the article, marking the issue as “solved” does not help me nor anyone that might read it in the future. cc’ing @Bianca Hoch @Jeff Hale
a
Hi @José Duarte! Excellent point regarding ✅ and we should probably clarify what that means. We use this emoji not to mark that some issue has been fully resolved on the user's end (only the user can judge that) but rather to signal to everyone else reading this thread that someone from the community or from the Prefect's team read this thread and provided enough level of guidance pointing the user in the right direction to solve their issue on their own. Ideally, if the question or issue is defined precisely enough and provides a Minimal Reproducible Example (MRE), we can help you solve the problem in a single response. Our community is growing so quickly that we don't have the capacity for detailed long 1:1 conversations - if you need that level of guidance, we offer a paid support service you can reach via cs@prefect.io. The scope of community support is meant to point the users in the right direction to solve their issues on their own, which is what the checkmark indicates. Thank you for asking this question. We should have clarified it before starting to use this emoji.
✅ 3
upvote 5
Regarding the error you are getting:
Copy code
MissingContextError: Logger 'dystematic.pdbt.ftp' attempted to send logs to Orion without a flow run id.
This shows that you are using a non-Prefect logger. Could you try switching to Prefect logger this way?
Copy code
from prefect import get_run_logger
logger = get_run_logger()
and regarding the root cause of your problem, I totally understand the pain and we address it with a context manager in this PR https://github.com/PrefectHQ/prefect/pull/6575 once this is merged, you should be able to disable Prefect logger in unit tests and other situations when you need to suppress it:
Copy code
import prefect
from prefect import task, flow, get_run_logger
from prefect.logging.loggers import disable_run_logger


@task
def your_task():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Prefect Version = %s", prefect.__version__)


@flow
def hi():
    your_task()


if __name__ == "__main__":
    with disable_run_logger():
        your_task.fn()
j
@Anna Geller I cannot switch as the FTP client is not supposed to depend on Prefect. I’ve since fixed the issue although I have no clue how. My guess was that there was an issue wrt to threading and switched the code to do the network calls from the main thread instead of a secondary one. That seems to have fixed but again, I have no clue why this fixed the issue.
a
thanks for the update and glad you managed to resolve it!
Open in Slack
PreviousNext
Powered by