Zhang David09/11/2023, 5:12 PM
the pool process, I can see the logs streamed in but mixed together across all flows so not really readable. My question is: is there any way that I could stream in the logs separately for each flow ? for me, this is very important for production monitoring and stuff. Really appreciate if anyone could help!
Islam Otmani09/14/2023, 9:58 PM
watch -n 1 'tail -n 15 /path/to/file_or_command | grep "run_name"'
Zhang David09/14/2023, 10:09 PM
watch -n 1 'tail -n 15 /path/to/file_or_command | grep "run_name"'In addition, is there any way I can stream the logs from a specific work queue from a work pool? I've actually being searching in the documentation, but couldn't find it.
and get the work pool process PID, and do a
tail -f /proc/$PID/fd/1
Islam Otmani09/14/2023, 10:10 PM
Zhang David09/14/2023, 10:11 PM
taking a look ^you mean the documentation above (in addition to the watch cmd) ?
Islam Otmani09/14/2023, 10:14 PM
if we can add the work queue name in there, so that you can grep as mentioned above
Zhang David09/14/2023, 10:17 PM
would not work - cuz my logs are only showing in console b/c its from my own library/pkg, so not from the prefect logger.
Islam Otmani09/14/2023, 10:32 PM
You can customize logging configuration by creating your own version ofAs I suspected, you're able to use thewith custom settings, by either creating the file at the default location (
logging.yml) or by specifying the path to the file with
/.prefect/logging.yml. (If the file does not exist at the specified location, Prefect ignores the setting and uses the default configuration.)
file to customize stuff. You can also configure handlers to write to files etc as needed, this is linked from our docs for further info: https://docs.python.org/3/library/logging.config.html#logging.config.dictConfig
Zhang David09/14/2023, 11:16 PM
Islam Otmani09/15/2023, 1:18 AM