What's a good way to try multiple model types with the meta-estimators (like
RandomizedSearchCV
)in
dask-ml
?
map
?
k
Kevin Kho
06/06/2022, 11:06 PM
Very naive about this, but doesn’t one meta estimator already use all available workers so you run into resource contention?
Kevin Kho
06/06/2022, 11:06 PM
Map feels like there’s two levels of abstraction if you know what I mean. It seems off because it should either be
map
or the
dask-ml
right?
m
Matt Alhonte
06/06/2022, 11:09 PM
Maybe! I dunno how it worker under the hood 😅 . Currently just using multiple Prefect tasks, which tbh is fine but I was wondering if I could avoid multiple loads of the data (and it's big enough that passing it through the Scheduler causes problems)
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.