https://prefect.io logo
Title
m

Manuel Aristarán

04/01/2020, 8:50 PM
Hi! Are there any examples of tasks that run in a Docker container, and capture its stdout?
j

josh

04/01/2020, 9:06 PM
Hey @Manuel Aristarán are you referring to a task which executes a Docker container like https://docs.prefect.io/core/task_library/docker.html#containers or do you mean having the tasks of your flow all run inside a Docker container?
m

Manuel Aristarán

04/01/2020, 9:08 PM
A single task which executes a container. I also need to attach volumes when creating a container, which
CreateContainer
does not seem to support
j

josh

04/01/2020, 9:10 PM
We always welcome contributions to the task library! As of right now I am not aware of any examples which do this but you could mimic the source code for the
CreateContainer
task https://github.com/PrefectHQ/prefect/blob/master/src/prefect/tasks/docker/containers.py#L8 and adjust it to fit your needs 🙂
m

Manuel Aristarán

04/01/2020, 9:10 PM
awesome, I might contribute support for the
volumes=
kwarg 🙂
thanks!
m

Manuel Aristarán

04/01/2020, 9:12 PM
yup
supporting
demux=
(
stdout
and
stderr
as separate results) might also be useful in
GetContainerLogs
j

josh

04/01/2020, 9:27 PM
Also another good suggestion! Quick Q: do you find the granularity of the Docker tasks to be good or cumbersome? For example it takes multiple tasks to create a container, start it, get the logs, and stop it. Whereas this could be accomplished in a single task.
m

Manuel Aristarán

04/01/2020, 9:44 PM
i’ll report back when I finish writing this flow 🙂 At first sight, yeah, it might be better —for one-off commands— to also have a
Task
that does everything