Thread
#prefect-ui
    Steven Hamblin

    Steven Hamblin

    1 year ago
    Hi there! Quick question: can someone help me understand why the Cloud UI shows me the
    .info
    log, but not the
    debug
    log in this snippet? (Using the logging example from the docs here: https://docs.prefect.io/core/concepts/logging.html)
    @task
    def my_task(data):
        from prefect import context
        logger = context.get("logger")
        <http://logger.info|logger.info>("info log inside mytask")
        logger.debug("debug log inside mytask")
    (I had thought that the Cloud UI was supposed to stream all logs including debug and I was meant to use the filters to reduce the clutter, but I’m having the opposite problem where for the life of me I can’t get debug logs to show up in the UI)
    If it’s relevant, I’m running this as a Docker flow.
    This is what I see in the logs:
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Hi Steven, the issue here is that the logs aren't being sent to cloud at debug level - you need to change the log level setting in the flow run to change this. There are a number of different ways to configure this (depending on your deployment setup), but the most straightforward way would be to pass the logging config as an environment variable to set on the flow run to your agent. Something like:
    prefect agent docker start -e PREFECT__LOGGING__LEVEL=DEBUG
    Note that this will affect all flows run by that agent.
    Steven Hamblin

    Steven Hamblin

    1 year ago
    Ah, that’s great, thanks Jim!
    @Jim Crist-Harif Curiously, that worked on my local Docker agent, but when I tried it with a Kubernetes agent, things didn’t go so well:
    prefect agent kubernetes install  -t $PREFECT_RUNNER_TOKEN --rbac | kubectl apply -f -          
    deployment.apps/prefect-agent created
    
    kubectl get pods                                                                                                            
    NAME                             READY   STATUS              RESTARTS   AGE
    prefect-agent-5b949974d8-5djw5   0/1     ContainerCreating   0          3s
    traefik-5b4c5db95c-pqztp         1/1     Running             0          7h21m
    
    kubectl get pods 
    NAME                             READY   STATUS    RESTARTS   AGE
    prefect-agent-5b949974d8-5djw5   1/1     Running   0          8s
    traefik-5b4c5db95c-pqztp         1/1     Running   0          7h21m
    
    kubectl delete deploy/prefect-agent                                                                                         
    deployment.apps "prefect-agent" deleted
    
    prefect agent kubernetes install -e PREFECT__LOGGING__LEVEL=DEBUG -t $PREFECT_RUNNER_TOKEN --rbac | kubectl apply -f -
    deployment.apps/prefect-agent created
    
    kubectl get pods                                                                                                            
    NAME                             READY   STATUS             RESTARTS   AGE
    prefect-agent-7ddc47c67c-wvsbn   0/1     CrashLoopBackOff   5          5m35s
    traefik-5b4c5db95c-pqztp         1/1     Running            0          7h28m
    
    kubectl logs pods/prefect-agent-7ddc47c67c-wvsbn                                                                            
    Traceback (most recent call last):
      File "/usr/local/bin/prefect", line 8, in <module>
        sys.exit(cli())
      File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in __call__
        return self.main(*args, **kwargs)
      File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main
        rv = self.invoke(ctx)
      File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
        return callback(*args, **kwargs)
      File "/usr/local/lib/python3.6/site-packages/prefect/cli/agent.py", line 267, in start
        start_agent(KubernetesAgent, **kwargs)
      File "/usr/local/lib/python3.6/site-packages/prefect/cli/agent.py", line 109, in start_agent
        agent.start()
      File "/usr/local/lib/python3.6/site-packages/prefect/agent/agent.py", line 219, in start
        self._verify_token(self.client.get_auth_token())
      File "/usr/local/lib/python3.6/site-packages/prefect/agent/agent.py", line 173, in _verify_token
        result = self.client.graphql(query="query { auth_info { api_token_scope } }")
      File "/usr/local/lib/python3.6/site-packages/prefect/client/client.py", line 286, in graphql
        raise AuthorizationError(result["errors"])
    prefect.utilities.exceptions.AuthorizationError: [{'path': ['auth_info'], 'message': 'AuthenticationError: Forbidden', 'extensions': {'code': 'UNAUTHENTICATED'}}]