Christopher Chong Tau Teng

12/02/2021, 11:42 AM
Hi @Anna Geller @Kevin Kho, according to the code structure here describing how to use Docker agent, I have some questions. Assuming the following is my directory
and samples from each file:
from datetime import timedelta
from prefect import Flow
from prefect.schedules import IntervalSchedule
from import GCS
from prefect.run_configs import DockerRun

import sys
from task_flow_1 import task_test_flow

schedule = IntervalSchedule(interval=timedelta(minutes=1))

with Flow("test-flow-1", schedule) as flow:
    task_test_flow() = GCS(bucket="xxx")
flow.run_config = DockerRun(image="xxx/prefect:v1")
import prefect
from prefect import task
import numpy as np

def task_test_flow():
    logger = prefect.context.get("logger")
    test_arr = np.array([1, 2, 3])
FROM prefecthq/prefect:latest
ADD . .
RUN prefect backend server
Now assuming I have registered both flows with the server and they are running as expected. One day suddenly,
breaks and I need to change
to fix the bug. I then updated the following image to v2 in
flow.run_config = DockerRun(image="xxx/prefect:v2")
I then built a new docker image v2 and pushed to
. Here’s my question: before I register these 2 flows with the server, do I also need to update the image in
to use
, or can it continue to use

Anna Geller

12/02/2021, 11:51 AM
@Christopher Chong Tau Teng #1 This line will cause you trouble:
Because even when you run your flow, it will then try to register the flow at flow runtime. It would be better to remove that line from the flow file and use the CLI instead for registration:
prefect register --project project-p
#2 When you use Docker storage, then your custom flow dependencies are packaged there. So you don’t need to have those dependencies installed on Server. Therefore, the image for Server can be a different Docker image than the one used for a flow. But if your tasks are defined outside of the flow file, then they won’t exist in the GCS storage. You would then have to rebuild the docker image any time you make any change to either your tasks or flow. To make it easier you could either: • change the image tag to latest so that it’s easier to update it on your run config • or perhaps switching to docker storage would make more sense in your use case? This way the image would always be built individually for each flow upon registration. Does it answer your question? I think the blog post you referenced was addressing the problem of packaging the custom module dependencies (i.e. packages), rather than packaging flow and task code. In your use case you would always need to rebuild the image so Docker storage makes actually more sense than GCS
:upvote: 1

Christopher Chong Tau Teng

12/06/2021, 3:06 AM
@Anna Geller thanks for your reply. Noted on #1. On #2, if we continue to choose to use GCS, then my question still stands - before I register these 2 flows with the server, do I also need to update the following in
flow.run_config = DockerRun(image="xxx/prefect:v2")
or can it continue to use 

Kevin Kho

12/06/2021, 3:45 AM
They are independent so it can continue to use
is there is no reason to upgrade if you don’t want to change it. No need to re-register also if you just want it to use
:upvote: 2

Christopher Chong Tau Teng

12/06/2021, 9:35 AM
cc. @Cormac Long