I'm facing an issue with my Prefect v3 deployment and was wondering if anyone has insights on where to start investigating. I recently set up a new deployment with a single worker running in process mode. The flow runs successfully, and I can see the logs most of the time. However, intermittently (roughly every 4th run), the logs are neither displayed in the UI nor saved to disk. This is quite strange since the other runs work perfectly fine. I cannot see any logs from Prefect server or worker when I see this issue, so I couldn't find any errors or unusual behavior that might explain this inconsistency. Does anyone have any leads on what could be causing this or suggestions on how I can debug this further? Any troubleshooting tips or similar experiences would be greatly appreciated! Thanks in advance! 🙏

Hey William! Whenever this happens, are the worker and flow run logs not visible in the stdout either? Usually I'd recommend enabling DEBUG level logging for something like this, although if the logs don't populate anywhere, it may not reveal much about what is happening 😅

Sorry, forget to reply here 👀 Yeap I tried with Prefect_LOGGING_LEVEL debug and other stuffs but I never see any logs. I can just see this logs from the first screenshot. But its "funny that" when I can see my logs, this first logs off the screenshot do not show