Looking to turn a scraper into a Prefect flow - anyone got a Dockerfile handy that configures Prefect + Selenium and forces them to play nicely? (no specific issues yet, I just know Selenium is fussy as hell)
k
Kevin Kho
05/30/2021, 10:32 PM
Hey @matta, I have used selenium but not with Prefect. I assume when you mean fussy, you mean including the webdriver right? That does seem like the trickier part of packaging the two. But I think as long as you download the webdriver and install it in the image, you should be fine?
m
matta
05/30/2021, 10:33 PM
Yeah, setting it up with a headless browser and everything.
matta
05/30/2021, 10:33 PM
Often requires very specific package versions (which sometimes break other things)
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.