<@ULVA73B9P> I will be creating an app that will s...
# ask-marvin
j
@Marvin I will be creating an app that will submit flows via a service account API key. How do I set configuration such as PREFECT_LOGGER_EXTRA_LOGGERS for a service account API key? Or, is that config handled inside the worker process (the flow will be executed via a local process worker)? And I'd set an environment variable inside the worker startup script?
m
thought for 424.7 seconds
Great question — and a common point of confusion. Short answer: Prefect settings are not tied to an API key. They’re loaded from the environment of the process that’s running. For a local process worker, that means the flow run subprocess needs to see the setting. A few key points: - The correct setting name in Prefect 3.x is PREFECT_LOGGING_EXTRA_LOGGERS (not PREFECT_LOGGER_EXTRA_LOGGERS). - Settings are evaluated in the process that runs the flow. With a local process worker, that’s the child process the worker spawns for each flow run. - Your service account API key just authenticates API calls; it does not carry logging/config settings. Ways to set it (pick what fits your setup): 1) Per-deployment (recommended for explicit, portable config) - In Python:
Copy code
from prefect import flow

@flow
def my_flow(): ...

my_flow.deploy(
    name="prod",
    work_pool_name="process-pool",
    job_variables={
        "env": {
            "PREFECT_LOGGING_EXTRA_LOGGERS": "boto3,botocore,urllib3",
        }
    },
)
- In prefect.yaml:
Copy code
deployments:
- name: prod
  entrypoint: flows/my_flow.py:my_flow
  work_pool:
    name: process-pool
    job_variables:
      env:
        PREFECT_LOGGING_EXTRA_LOGGERS: "boto3,botocore,urllib3"
2) At the work-pool template (applies to all deployments using that pool) - Put the env in the pool’s base job template (for process workers, the
env
field). This is useful if you want the same setting everywhere. See “Customize job variables” docs below. 3) On the worker host (quick and simple, inherited by flow runs) - In the script/service that starts your worker:
Copy code
export PREFECT_API_KEY=***   # service account
export PREFECT_LOGGING_EXTRA_LOGGERS="boto3,botocore,urllib3"
prefect worker start -p process-pool
This will typically be inherited by the flow run subprocess. Using job_variables (options 1 or 2) is more explicit and survives worker restarts or different machines. Related settings you might care about: - PREFECT_LOGGING_LEVEL - PREFECT_LOGGING_LOG_PRINTS - PREFECT_LOGGING_TO_API_ENABLED (defaults to true; logs will go to the API) Docs for deeper reference: - Logging customization - Settings and profiles (precedence) - Customize job variables - Process worker API reference If you share how you’re managing deployments (prefect.yaml vs Python
deploy()
), I can give the exact snippet you need.