Vikram Iyer

06/29/2020, 5:09 PM
Hey Guys, I have recently started evaluating prefect for my ML work at my current company. I have a few questions around "how to" use prefect for my local deployment. Setup:- I have a backend service that runs inside a docker container on a host machine. Prefect server (prefect server start) and agent (prefect agent start) run on the same host. Question:- How do I register flows from within my container to the server on host and make them run from UI? So, when the service is running on the host I can simple do the below steps
1. flow.register() # this will register the flow and hence I can see it on the UI
2. From the UI, I can go to the particular flow and run it. # easy
How do I do something similar, with the only exception of the service and code residing inside a docker container? End Notes:- I might be looking at it from a completely wrong perspective, please correct my understanding if necessary. Cheers! Vikram

Kyle Moon-Wright

06/29/2020, 7:52 PM
Hey @Vikram Iyer, Apologies for the late reply. These are great questions, I'll do my best to provide some direction here. You can most certainly use Prefect with the Server UI residing in a container, however the containerized code you are using will have to be stored - typically users will utilize a container registry for this. When the agent picks up the flow from the UI, it'll pull the stored container to execute in the agent's environment, using whatever environment configurations you may have set. All of this will be configured by running your flow with
which will provide your agent the flow configurations and allow you to start flow runs from the UI.

Vikram Iyer

06/30/2020, 4:08 AM
This is awesome, thanks @Kyle Moon-Wright, let me try this. I will reach out of I bump onto any roadblocks. 🙂 Meanwhile, I figured out, we can use a docker-compose that can bring up entire prefect core along with any additional services I require to run my workflows. Since they are in the same network, there shouldn't be any problem of storage as long as the code resides on the agents.
👍 1