Is it possible to run tasks in docker environment?
Usecase:
I have a Flow composed of 3 tasks. First two are pretty simple, and can be done without using any external python library. The third one requires some external python libraries and a few system dependencies to be installed. Is there a way to run the first two tasks on the host environment, and the third in docker?
Since you can only deploy a flow, not a task, you can’t run tasks in different environments. However, you can use subflow to do the same thing
p
Prakash Rai
08/31/2022, 4:03 PM
Hi @Khuyen Tran Thanks for the prompt reply. However, this doesn't solves the problem I'm facing.
Here's my question in context of subflows: Can I deploy the main flow in one environment, and the subflow in another? I want to invoke my main flow in host environment, but my subflow should be invoked in
docker
environment.
k
Khuyen Tran
08/31/2022, 6:02 PM
You could have different subflows in different environments. For your use case, you can run the first two subflows on the host environment, and the third in docker
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.