Denis
11/28/2022, 4:10 PM# len(param_dict) => 100
for params in param_dict:
copy_files.submit(params)
will this create 100 threads or is it using async to run it concurrently in one thread. The reason I am asking is because we're facing issues of hanging flown runs when we submit a larger number of tasks, (in last attempt we had cca 48 tasks and flow just hanged)
Any clarification on how it works would be appreciated.Jeff Hale
11/28/2022, 4:24 PMDenis
11/28/2022, 6:07 PMJeff Hale
11/28/2022, 10:14 PMZanie
11/28/2022, 10:17 PMDenis
11/29/2022, 8:31 AMCrash detected! Execution was interrupted by an unexpected exception: PrefectHTTPStatusError: Client error '408 Request Timeout
Any idea? @ZanieZanie
11/29/2022, 3:19 PMDenis
11/29/2022, 3:24 PMCrash detected! Execution was interrupted by an unexpected exception: PrefectHTTPStatusError: Client error '408 Request Timeout' for url '<https://api.prefect.cloud/api/accounts/60a39fa6-c00d-4e61-8757-a9c1fa7b6f11/workspaces/245aa934-2480-45c4-845e-2f66f1cdf298/task_runs/f533b916-97b3-46d3-88f8-b3a6273a7781/set_state>'
Zanie
11/29/2022, 3:26 PMDenis
11/29/2022, 3:55 PMalvin goh
11/30/2022, 6:40 AMVadym Dytyniak
12/01/2022, 7:57 AMZanie
12/01/2022, 2:59 PMVadym Dytyniak
12/01/2022, 3:01 PMKelvin DeCosta
02/09/2023, 11:10 AM408 Request Timeout
errors.
Just like alvin, I'm wondering whether it could be due to us being on the free plan.Zanie
02/09/2023, 2:29 PMZach Angell
02/09/2023, 2:51 PMtime.sleep
) that doesn’t allow our asynchronous client to send data in a timely mannerKelvin DeCosta
02/10/2023, 6:45 AMrequests
(as opposed to httpx
) could be a reason for this error. Zach mentioned above that this could happen when:
Flow code is performing excessive blocking work (e.g.Do you think this could be the case?) that doesn’t allow our asynchronous client to send data in a timely mannertime.sleep
Zanie
02/17/2023, 3:07 PM