I've got a Prefect 3.1.5 instance setup in Kuberne...
# ask-community
d
I've got a Prefect 3.1.5 instance setup in Kubernetes and my jobs are dying right after starting complaining about
PREFECT_LOGGING_SERVER_LEVEL
- I'm not sure why the 'jobs' which are started by the 'worker' would be complaining about the 'server' logging level, any ideas? (traceback in the thread)
Copy code
kubectl logs pod/prefect-job-w4srl-zn2pf

Traceback (most recent call last):
  File "/usr/local/bin/prefect", line 5, in <module>
    from prefect.cli import app
  File "/usr/local/lib/python3.11/site-packages/prefect/cli/__init__.py", line 12, in <module>
    import prefect.cli.shell
  File "/usr/local/lib/python3.11/site-packages/prefect/cli/shell.py", line 16, in <module>
    from prefect import flow
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/usr/local/lib/python3.11/site-packages/prefect/__init__.py", line 117, in __getattr__
    module = import_module(module_name, package=package)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/main.py", line 55, in <module>
    prefect.logging.configuration.setup_logging()
  File "/usr/local/lib/python3.11/site-packages/prefect/logging/configuration.py", line 71, in setup_logging
    config = load_logging_config(
             ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/logging/configuration.py", line 41, in load_logging_config
    template.substitute(current_settings.to_environment_variables())
  File "/usr/local/lib/python3.11/string.py", line 121, in substitute
    return self.pattern.sub(convert, self.template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/string.py", line 114, in convert
    return str(mapping[named])
               ~~~~~~~^^^^^^^
KeyError: 'PREFECT_LOGGING_SERVER_LEVEL'
n
hi @Devin Flake - are you using a
logging.yaml
? it looks like there's potentially some extra incorrect key in that file
if you do have that key set in there, try switching it for
PREFECT_SERVER_LOGGING_LEVEL
d
ah - yes I do, I'll give that a try! Thanks Nate!
👍 1
@Nate quick clarification, so in
logging.yaml
it should be
PREFECT_SERVER_LOGGING_LEVEL
but for the Prefect Server the env variable is
PREFECT_LOGGING_SERVER_LEVEL
?
n
I think it should be
SERVER_LOGGING
either way, I think we accept
PREFECT_LOGGING_SERVER_LEVEL
as an alias for the settings object (and we should also accept it as an alias in the yaml, but it seems we dont)
d
got it - thanks!
fyi - that worked, I also had to change
PREFECT_LOGGING_INTERNAL_LEVEL
to
PREFECT_INTERNAL_LOGGING_LEVEL
n
👍