Another pretty low-level question: I'm debugging a...
# prefect-contributors
Another pretty low-level question: I'm debugging a custom Environment and would like to: 1. Run with local agent (that part works fine, can register a local flow & have it run, etc.) 2. See debug logging during the Flow run / Environment execute(). I can enable & see debug output for the local agent, but I think since the flow run happens in a subprocess all I get is:
Copy code
[2020-04-30 20:35:05,397] INFO - agent | Found 1 flow run(s) to submit for execution.
[2020-04-30 20:35:05,397] DEBUG - agent | Updating states for flow run f7912fd5-8ce3-4048-8cfe-965a15d46845
[2020-04-30 20:35:05,400] DEBUG - agent | Flow run f7912fd5-8ce3-4048-8cfe-965a15d46845 is in a Scheduled state, updating to Submitted
[2020-04-30 20:35:05,505] INFO - agent | Deploying flow run f7912fd5-8ce3-4048-8cfe-965a15d46845
[2020-04-30 20:35:05,519] DEBUG - agent | Submitted flow run f7912fd5-8ce3-4048-8cfe-965a15d46845 to process PID 46552
[2020-04-30 20:35:05,608] DEBUG - agent | Completed flow run submission (id: f7912fd5-8ce3-4048-8cfe-965a15d46845)
[2020-04-30 20:36:22,277] INFO - agent | Process PID 46552 returned non-zero exit code
Is there a good way to (a) have the flow runner in process or (b) see log output from the subprocess?
FWIW, the new Environment is working great but the project it's using (Dask Cloud Provider) is doing some weird things where it deregisters Fargate tasks for tasks that it didn't create... %$#*@)
shaking angry fist 1
I believe if you start the agent with the
you should see the flow run logs in the agent process
Oooo, yes! Trying...
Bam, works! Thank you! (Yet again 🙂)
Awesome! anytime 😄
@Chris White sorry, one more (hopefully) quick thing. Do you know if it's possible to (easily) set the root logging level for the flow run? (I see the Prefect logging level is set to debug here: but I want to get log output for e.g.
Popen is calling
prefect execute cloud-flow
(Actually, let me just futz with the root logger level in my Environment's execute()... nm)
(That worked)
Hmmmm are you trying to set a universal root level for all python loggers?
Yeah, or ideally control the levels for specific loggers, e.g. "botocore & urllib3 are too chatty so leave them at INFO but set root to DEBUG", etc. I was able to do that at the start of my Environment's execute() function & all worked well. Thanks!
For sure! yea we don’t expose a native hook for configuring other loggers but it’s definitely an interesting idea; if they can be configured through environment variable then you can use
prefect agent start -e LOGGING_ENV_VAR=val
to pass the environment variable to each flow run, but whether that works depends on the package probably
👍 1