Toby Rahloff
09/23/2022, 2:20 PMaddress
param removed from the code snippet for readability).
It seems like resetting the Prefect database fixes this issue. Could the tracking of tasks overload the DB? Is this edge-case known and are there workarounds for it?Jeff Hale
09/23/2022, 2:21 PMToby Rahloff
09/23/2022, 2:26 PMpip freeze
and poetry pyproject.toml belowToby Rahloff
09/23/2022, 2:27 PMToby Rahloff
09/23/2022, 2:27 PMJeff Hale
09/23/2022, 2:28 PMToby Rahloff
09/23/2022, 2:30 PMfailed to parse annotation from 'Name' node: 'NoneType' object has no attribute 'resolve'
bug reported by Oscar Bjƶrhn for me (Slack ref. )Toby Rahloff
09/23/2022, 2:31 PMJeff Hale
09/23/2022, 3:24 PMfailed to parse annotation from 'Name' node: 'NoneType' object has no attribute 'resolve'
should be forthcoming very soon.
The improvements in 2.4.1 (2.4.2 presumably after the fix above) might eliminate the issue totally for you, but I think thatās correct.Toby Rahloff
09/23/2022, 3:37 PMZanie
Zanie
prefect version
?Toby Rahloff
09/23/2022, 4:17 PM⯠prefect --version
2.4.1
Jeff Hale
09/23/2022, 4:21 PMprefect version
From the cli, no dashes, should give a bunch more output.Toby Rahloff
09/23/2022, 4:21 PMtime.sleep
), Ray outperformed Dask by far due to the memory congestion that happens on the Dask Head Node when managing over ~1000 tasks.
It also outperformed the ConcurrentTaskRunner in our tests.
"Weāre working with the Ray team" that is amazing to hear! Is there a way to follow this effort? Please also let me know if we can support this in any wayToby Rahloff
09/23/2022, 4:21 PM⯠prefect version
Version: 2.4.1
API version: 0.8.0
Python version: 3.8.10
Git commit: e941523d
Built: Thu, Sep 22, 2022 12:26 PM
OS/Arch: linux/x86_64
Profile: default
Server type: ephemeral
Server:
Database: sqlite
SQLite version: 3.31.1
Zanie
Zanie
prefect orion start
) and use Postgres once youāre at scale (we have many tasks performing writes in parallel with separate clients which is not SQLiteās forte).Toby Rahloff
09/23/2022, 4:23 PMToby Rahloff
09/23/2022, 4:23 PM⯠prefect version
Version: 2.4.1
API version: 0.8.0
Python version: 3.8.10
Git commit: e941523d
Built: Thu, Sep 22, 2022 12:26 PM
OS/Arch: linux/x86_64
Profile: dev
Server type: cloud
Zanie
Can you share under which circumstances Ray is the slowest TaskRunner?Thatās interesting to hear! I just noticed it was slow when I was building it out.
Zanie
Zanie
Toby Rahloff
09/23/2022, 4:24 PMToby Rahloff
09/23/2022, 4:24 PMZanie
Zanie
Zanie
Toby Rahloff
09/23/2022, 4:28 PMZanie
Zanie
Toby Rahloff
09/23/2022, 4:39 PMZanie
asyncio.gather
or anyio.TaskGroup
)Zanie
Toby Rahloff
09/23/2022, 4:48 PMZanie
sleep(1)
theyāll have more spread out requests for orchestration š itās weird that the flow run is freezing though, thereās no case where that should be happeningZanie
Toby Rahloff
09/26/2022, 7:29 AM