What's a good way to try multiple model types with the meta-estimators (like
RandomizedSearchCV
)in
dask-ml
?
map
?
k
Kevin Kho
06/06/2022, 11:06 PM
Very naive about this, but doesn’t one meta estimator already use all available workers so you run into resource contention?
Kevin Kho
06/06/2022, 11:06 PM
Map feels like there’s two levels of abstraction if you know what I mean. It seems off because it should either be
map
or the
dask-ml
right?
m
Matt Alhonte
06/06/2022, 11:09 PM
Maybe! I dunno how it worker under the hood 😅 . Currently just using multiple Prefect tasks, which tbh is fine but I was wondering if I could avoid multiple loads of the data (and it's big enough that passing it through the Scheduler causes problems)