Hi. I continue to ask my questions here - super grateful for the help i got so far. I have now setup a flow that launches a number of subflows that fetches data from Google Cloud Storage and writes the data to Big Query. I’m running the flow locally (prefect 2.0) and it’s connected to prefect cloud. However, i sometimes run into an error i cannot really understand and i don’t know how to troubleshoot it. It only happens occasionally and i cannot really reproduce it. I moved the trace to the thread. The logging message @

16:34:10.826

is the last thing that happens in the task read_blob in which the error occurs in. So it looks to me that something goes wrong when reporting the task. The error message doens’t tell me that much - so if you have any advices on how i should troubleshoot this i would be really helpful (or if you can guess on what might be wrong?)

can you move the traceback into the thread?

The details of a crashed task run will be included as a debug level log

Thank you both for your feedback. Realize that i might revisit how i defined the task - to see if that make things run smoother. And thanks for the advice regarding log level - i will change! If post the stack trace for reference in the thread: I get the following error message:

16:34:10.826 | INFO    | Task run 'read_blob-eaa21aac-2' - Fetched z8hobwiqtxc0_2022-05-20T110000_adf3949b6711af3e17e519b166d653ee_c3575d.csv.gz - 57334 rows
16:34:19.665 | INFO    | Task run 'read_blob-eaa21aac-2' - Crash detected! Execution was interrupted by an unexpected exception.
16:34:29.330 | ERROR   | Flow run 'myrtle-millipede' - Encountered exception during execution:
Traceback (most recent call last):
  File "/Users/daniel/Library/Caches/pypoetry/virtualenvs/marketingdata-pZAGOfI1-py3.10/lib/python3.10/site-packages/prefect/engine.py", line 470, in orchestrate_flow_run
    await wait_for_task_runs_and_report_crashes(
  File "/Users/daniel/Library/Caches/pypoetry/virtualenvs/marketingdata-pZAGOfI1-py3.10/lib/python3.10/site-packages/prefect/engine.py", line 882, in wait_for_task_runs_and_report_crashes
    result = await client.set_task_run_state(
  File "/Users/daniel/Library/Caches/pypoetry/virtualenvs/marketingdata-pZAGOfI1-py3.10/lib/python3.10/site-packages/prefect/client.py", line 1688, in set_task_run_state
    response = await <http://self._client.post|self._client.post>(
  File "/Users/daniel/Library/Caches/pypoetry/virtualenvs/marketingdata-pZAGOfI1-py3.10/lib/python3.10/site-packages/httpx/_client.py", line 1820, in post
    return await self.request(
  File "/Users/daniel/Library/Caches/pypoetry/virtualenvs/marketingdata-pZAGOfI1-py3.10/lib/python3.10/site-packages/httpx/_client.py", line 1506, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/Users/daniel/Library/Caches/pypoetry/virtualenvs/marketingdata-pZAGOfI1-py3.10/lib/python3.10/site-packages/prefect/client.py", line 233, in send
    response.raise_for_status()
  File "/Users/daniel/Library/Caches/pypoetry/virtualenvs/marketingdata-pZAGOfI1-py3.10/lib/python3.10/site-packages/httpx/_models.py", line 1510, in raise_for_status    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url '<https://api-beta.prefect.io/api/accounts/df91f491-10a1-4937-abc9-fd13842df41f/workspaces/caa7e1bf-faa2-4ba5-baab-1987efa226f1/task_runs/ad18c70f-1c42-45af-8655-a3143bc874e7/set_state>'
For more information check: <https://httpstatuses.com/500>
16:34:35.466 | ERROR   | Flow run 'myrtle-millipede' - Finished in state Failed('Flow run encountered an exception.')