Hi there slightly smiling face One question about dynamic fl

Question

Hi there slightly smiling face One question about dynamic flows build at runtime and the prefect server cloud Our ETL process contains entities to load stored in the database The list of these entities can change over time because of development How can I achieve a dynamic load which will load all entities existing at runtime and be able to manage this via the Server Cloud I can not use the task mapping function because the entities have own dependencies also stored in the database and these will be dynamically resolved at runtime Is there a way to solve this kind of problem with prefect UI I developed this with the localDaskRun and the explicit run call Then its working because the flow is built everytime Now via Server I got errors when the task is changing ```KeyError Task slug XX not found in the current Flow this is usually caused by changing the Flow without reregistering it with the Prefect API ``` Best Michael

Michael Hadorn · Answer

When I try to use the gitlab storage for this dynamic flow I got this error ```raise ValueError No flow found in file ``` But for sure this flow is built in this file The same file is working without when I use the localStorage instead I think I have a general problem because my implementation of the flow is they way it should

josh · Answer

Hi < Michael Hadorn> when using the GitLab storage are you also manually committing the file to the git repo Git based storage does not auto upload the flow file to the location at build time like some of the other storage options

Michael Hadorn · Answer

yes the same is working with the easy flow

Michael Hadorn · Answer

and I use the same repo like i use to register the flow initially

Michael Hadorn · Answer

is there somewhere a explanation what with the files in the storage exactly happens is the flow rebuilt when the flow is stored as file what then will be registered in the gui the same output of the flow

Michael Hadorn · Answer

ok it s working now i figured out how the file from the storage is parsed <https github com PrefectHQ prefect blob cbb2040b8f1375e4a55a274f8fbadb02990cf2fa src prefect utilities storage py L91> then i had to look that this is available gt right position of flow not indented to solve the other issue with dynamic flow gt i will create another flow which is rebuilding registering the other one

Michael Hadorn · Answer

Because i don t think that dynamic flows will work like I would like sweat smile