https://prefect.io logo
Title
p

Prakash Rai

08/31/2022, 2:57 PM
Is it possible to run tasks in docker environment? Usecase: I have a Flow composed of 3 tasks. First two are pretty simple, and can be done without using any external python library. The third one requires some external python libraries and a few system dependencies to be installed. Is there a way to run the first two tasks on the host environment, and the third in docker?
1
The link here https://docs.prefect.io/tutorials/docker/#prerequisites explains how to run a flow in a docker environment, but talks nothing about task level docker support
k

Khuyen Tran

08/31/2022, 3:33 PM
Since you can only deploy a flow, not a task, you can’t run tasks in different environments. However, you can use subflow to do the same thing
p

Prakash Rai

08/31/2022, 4:03 PM
Hi @Khuyen Tran Thanks for the prompt reply. However, this doesn't solves the problem I'm facing. Here's my question in context of subflows: Can I deploy the main flow in one environment, and the subflow in another? I want to invoke my main flow in host environment, but my subflow should be invoked in
docker
environment.
k

Khuyen Tran

08/31/2022, 6:02 PM
You could have different subflows in different environments. For your use case, you can run the first two subflows on the host environment, and the third in docker