Zach12/08/2020, 6:39 PM
Jim Crist-Harif12/08/2020, 7:00 PM
with workers running in individual containers, and use resource labels to lock tasks to run in a specific worker. This is a lot more complicated setup (a lot more complicated), but could also be useful in some scenarios. If possible, I recommend the first option. We could add a new task to the task library to create, run, and wait on a single docker container that would make this pattern easier, but it would still be for executing a CLI command inside a container not running a python function.
Zach12/08/2020, 7:17 PM
Jim Crist-Harif12/08/2020, 7:27 PM
It would also be helpful to have a way to copy a file into the docker containerThat seems like a bit of scope creep, I think for that I'd want the user to write their own custom thing - prefect works fine with user-written tasks, only common patterns make sense for inclusion in the task library.
Zach12/08/2020, 7:37 PM
Jim Crist-Harif12/08/2020, 7:42 PM
Zach12/08/2020, 10:34 PM
Jim Crist-Harif12/08/2020, 11:02 PM