https://prefect.io logo
Title
h

Horatiu Bota

06/03/2022, 4:55 PM
hi community! i'm following this to add extra loggers for my runs, but the log messages are not showing up on prefect server, anyone have similar issues?
k

Kevin Kho

06/03/2022, 5:08 PM
Aren’t you using 2.0? Or did you move to 1.0?
h

Horatiu Bota

06/03/2022, 5:08 PM
using 1.0, haven't moved to 2.0 yet - i saw that the syntax for 2.0 is different
k

Kevin Kho

06/03/2022, 5:13 PM
How are you using it? Adding it to the RunConfig or config.toml?
h

Horatiu Bota

06/03/2022, 5:14 PM
adding it to run config right now - would like to add it to the prefect server helm chart if possible
k

Kevin Kho

06/03/2022, 5:15 PM
RunConfig is a good text and guaranteed to work. I don’t see how you can add it to the server helm chart unless you deploy your agent with it? You can add it as an env var on the agent to be carried over to the Flow
h

Horatiu Bota

06/03/2022, 5:16 PM
yep, i'm deploying the agent (KubernetesAgent) with the chart
k

Kevin Kho

06/03/2022, 5:19 PM
The syntax should be like this
- name: PREFECT__CLOUD__AGENT__ENV_VARS
          value: '{"PREFECT__LOGGING__EXTRA_LOGGERS": "['boto3', 'botocore']"}'
You can find it by typing
prefect agent kubernetes install --env PREFECT__LOGGING__EXTRA_LOGGERS="['boto3', 'botocore']"
And see where it adds it in the template
☝️ 1
h

Horatiu Bota

06/03/2022, 5:53 PM
thanks! currently i'm just trying to get it working via the run config, but nothing seems to work
k

Kevin Kho

06/03/2022, 5:55 PM
Custom library or something like boto3?
h

Horatiu Bota

06/03/2022, 5:55 PM
custom library - my run config looks like:
KubernetesRun(env={'PREFECT__LOGGING__LEVEL': 'DEBUG', 'PREFECT__LOGGING__EXTRA_LOGGERS': '["custom"]'})
(tried swapping the quote/apostrophe but that didn't work)
fyi running locally logs fine, on the k8s run logs are not showing up
k

Kevin Kho

06/03/2022, 6:07 PM
I have a working example here for custom
h

Horatiu Bota

06/03/2022, 6:10 PM
thanks! that's pretty much what i'm doing (working on a very simple example) except i'm setting the
PREFECT__LOGGING__EXTRA_LOGGERS
env variable instead of setting the config.toml value
flow.run() locally works ok, it's the k8s run that doesn't show the logs in the prefect server ui
also, setting
"PREFECT__LOGGING__LEVEL": "DEBUG"
in the run config works fine (i.e., sets the right level), it's just the extra loggers that don't
k

Kevin Kho

06/03/2022, 7:42 PM
I think local is not a good representation of what happens in the Cloud for this kind of thing because stdout can be captured when using
flow.run()
whereas this logger needs the CloudHandler attached to the logs to send those. I think the logger might not be configured right? Maybe you can try with the test code above?