Just published a blog about connecting Prefect to ...
# show-and-tellr
Richard Pelgrim
09/24/2021, 2:14 PMJust published a blog about connecting Prefect to a Dask cluster using Coiled for processing larger-than-memory Tasks and/or Flows.
https://towardsdatascience.com/scaling-your-prefect-workflow-to-the-cloud-2dec4e0b213b
👏 7
🚀 15
marvin 8
i
itay livni
09/27/2021, 2:39 PM@Richard Pelgrim Nice. I like the resource manager example...
a
Andrew Black
09/27/2021, 4:49 PMExcellent, Richard! I really like how you describe what Prefect does, how our technologies work together, and the TLDR at the top.
r
Richard Pelgrim
09/28/2021, 6:47 AMThanks @itay livni - lmk if you end up implementing it with Coiled anywhere. Would love to have a look!
Richard Pelgrim
09/28/2021, 6:48 AMThanks @Andrew Black, happy to read you enjoyed the post!
t
Todd de Quincey
11/06/2021, 7:53 AMGreat example gist. Exceptionally clear, concise and covers all the bases. Awesome work @Richard Pelgrim
r
Richard Pelgrim
11/06/2021, 1:04 PMThanks @Todd de Quincey! 🙏 would love to hear about anything (good or bad) you're running into with Dask clusters!
t
Todd de Quincey
11/06/2021, 2:48 PMWill do. Just looking at coiled. What’s included in the free tier? 1000 CPU hours per month if I am reading it correctly
r
Richard Pelgrim
12/17/2021, 2:52 PMSlack just sent me a notification for this message... sorry to have missed this earlier!
How was your Coiled onboarding?
t
Todd de Quincey
12/17/2021, 2:54 PMDidn't use it lol
r
Richard Pelgrim
12/17/2021, 3:29 PMhaha, ok, fair enough 🙂
Richard Pelgrim
12/17/2021, 3:29 PMbecause it was tricky onboarding or because you didn't end up needing it?
3 Views