Felix Sonntag08/08/2022, 2:11 PM
and it works, but for example it does not cover logs, like coming from the Tensorflow CPP logs, as e.g.:
Plus, it’s kind of tedious collecting all the libs I want to include. 2. When starting a flow run, many valuable logs and especially errors are only visible on the work queue. So if e.g. I misconfigured a block and the deployment run fails to start, an end user cannot see the error message, since the queue logs are not visible. E.g. one would need to check the work queue Kubernetes pod and manually filter out the error logs for yourself. A user might not have access and with many flow runs, this is also hard. Am I missing something, or how should one work with that properly?
2022-08-08 13:59:27.431966: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2022-08-08 13:59:27.432037: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Khuyen Tran08/08/2022, 2:50 PM
Felix Sonntag08/09/2022, 1:56 AM
Anna Geller08/10/2022, 4:22 PM
prefect config set PREFECT_LOGGING_EXTRA_LOGGERS='tensorflow,anotherone'
Felix Sonntag08/11/2022, 7:51 AM
Anna Geller08/11/2022, 10:08 AM
Felix Sonntag08/11/2022, 10:26 AM
Anna Geller08/11/2022, 10:40 AM
Felix Sonntag08/11/2022, 10:50 AM
Anna Geller08/11/2022, 10:53 AM
Felix Sonntag08/11/2022, 10:56 AM