https://prefect.io logo
Title
f

Federico Zambelli

04/21/2023, 2:52 PM
I achieved so much concurrency it became a singularity 😂 Ninja edit: for anyone curious, code is here. I'm downloading about a year of r/dataengineering subreddit data using Reddit API with asyncpraw. https://github.com/wtfzambo/de_zoomcamp_project/tree/main/prefect
😂 8
🤯 3
❤️ 9
b

Brandon Reid

04/21/2023, 3:27 PM
ugg, I gotta get the x/y independent scaling going! haha
f

Federico Zambelli

04/21/2023, 3:46 PM
yeah I guess in this case no amount of scaling would make it readable, it's something like 1500 tasks at once 😄
🔥 2
j

Jeremiah

04/22/2023, 6:58 PM
On behalf of @Brandon Reid: challenge accepted 😂
👍 1