While trying to set log format in kubernetes for the schedul

Question

While trying to set log format in kubernetes for the scheduler I ve tried setting it like this ``` name PREFECT LOGGING FORMAT value level levelname s message message s ``` And I can see the pod has the environment variable set ```kubectl exec it prefect scheduler 6b96994c6 qtqh5 namespace=data bin sh c echo password $PREFECT LOGGING FORMAT lt aws default gt password level levelname s message message s ``` But the logs are still in the default format ```kubectl namespace data logs prefect scheduler 6b96994c6 qtqh5 f lt aws default gt 2020 06 09 11 36 54 302 INFO prefect server Scheduler | Scheduler will start after an initial delay of 275 seconds ``` Does anyone have any idea Or even better I m trying to get logs in datadog to be parsed correctly

Ben Davison · Answer

Just realised is `PREFECT LOGGING FORMAT` just for tasks outputs

Ben Davison · Answer

ah it s `PREFECT SERVER LOGGING FORMAT`

Ben Davison · Answer

Interesting even with ``` name PREFECT SERVER LOGGING FORMAT value level levelname s message message s ``` The output is blank ```kubectl namespace data logs prefect scheduler 649d8c4cb5 mz2pl f lt aws default gt ^C``` edit deffo escaping issue

Ben Davison · Answer

Strange ``` name PREFECT SERVER LOGGING FORMAT value level levelname s message message s ``` Will show the logs with I ve tried escaping like so ``` name PREFECT SERVER LOGGING FORMAT value level levelname s message message s ``` Same and if I only do single no logs show up atall

Ben Davison · Answer

yup will try but I think it will have the same escaping issue

Zachary Hughes · Answer

Sorry about that typed that before coffee and looks like you ve already given it a shot Taking a look into this as well

Zachary Hughes · Answer

One suggestion and apologies if you ve already tried it can you try specifying the logging format like this ``` name PREFECT SERVER LOGGING FORMAT value level name | message ``` It looks like there might be a double quoting issue in the snippets you posted

Ben Davison · Answer

yup that works but I m trying to get it to output json so datadog can parse the logs easily

Ben Davison · Answer

ah didn t see the remove s part

Ben Davison · Answer

ok I m convinced it s the ` ` Doesn t work `value level levelname s ` Works but has the double issue `value level levelname s ` I ve tried escaping both the quotes with ` ` and the ` ` also It always seems to double up whatever I put there

Zachary Hughes · Answer

I m seeing some weird behavior here but I m not sure that it s related to the If I provide the value below which was a typo I get what I think you re looking for with one trailing quote on the message value If I remove the typo I don t see any logs ` level levelname s message message s ` I m going to poke around for just a bit more but will open an issue if I don t find a solution in the near future It seems to be related to parsing for the last key will keep you posted

Ben Davison · Answer

thanks Zachary

Zachary Hughes · Answer

< Marvin> open Issues specifying JSON log formatting

Marvin · Answer

Zachary Hughes · Answer

Issue opened I think there is something hinky going on so this will let us triage and track a fix Thank you for letting us know about this < Ben Davison>