https://prefect.io logo
#prefect-community
Title
# prefect-community
m

Matthew Seligson

05/26/2022, 4:11 PM
When using the GitLab storage, in what scenarios is the flow code pulled? Is there caching? My assumption is that flow code is pulled from gitlab only upon initial start and restart from a failed state. Can the flow code be pulled while some tasks are running?
k

Kevin Kho

05/26/2022, 4:13 PM
Exactly as you describe. Not while tasks are running
m

Matthew Seligson

06/03/2022, 7:26 PM
https://docs.prefect.io/core/idioms/script-based.html This seems to indicate that flow code can be updated while tasks are running. Can you help me reconcile this?
k

Kevin Kho

06/03/2022, 7:39 PM
More like, if you edit the stuff inside a task
Copy code
@task
def abc():
    ....
That can take effect next flow run. It won’t take affect for the current flow run because the first step to start a flow is loading in and reading the script. So basically it’s the same as changing Python script code after a Python script is running. It already started the process so it won’t change
m

Matthew Seligson

06/03/2022, 7:42 PM
If you restart a flow run, it will call get_flow() on the storage object right? So it will load potentially new script code
k

Kevin Kho

06/03/2022, 7:44 PM
That’s true because we don’t cache any code on our side. It will raise an error if the DAG structure itself has changed, but if it’s an edit within a Task code, Prefect doesn’t look into that
m

Matthew Seligson

06/03/2022, 7:46 PM
Right. If I have a running flow with task A running some parallel task B in a failed state, then I change B’s task code and then restart the flow from B, will B load the new code?
Every time you restart a flow, even if it is currently running, won’t it start up a new pod (if using a kubernetes runner)?
k

Kevin Kho

06/03/2022, 8:05 PM
I believe that is right. It will load the new code. Can test in a bit if it helps you. I believe it will be true for all agents
10 Views