Hi all, Bear with me on this, but I'm a pretty new data engineer with a few Prefect flows under my belt, and have a question on best practices with regards to model deployment: Current setup is this: Agent is on AKS, file storage on Blob (we're an Azure shop). Using Prefect Cloud. What I'm trying to do: I have a model that has a child folder called Data. I'm trying to run this model using Prefect. Basically, I need to call a few API's and save the .csv's in the Data folder, run the model that takes input from the newly saved files, output some new files into the Data folder, and then I need to do some other transformations on a couple of the outputted files prior to snowflake upload. I've so far thrown the entire model folder system into the Flows folder, deployed the flow to Blob Storage, and attempted to run the model using a

poetry run python src/driver/__main__.py

shell_run_command. This has been a slow process as there's been a bunch of little issues arising, etc. My question: What would be the easiest way of doing all this? ie. what would be the best method to approach this problem? I feel like my method is a bit... patchy?

Hi @Amir, I wonder what are the little issues you encountered?

Hi @Khuyen Tran! Little issues were not related to Prefect; they pertained to issues I'd come across trying to get everything set up using the aforementioned method without having a lot of resources to build off of. The issues have (so far) been navigable! There is a folder that contains all the code for the model. It looks something like this: - MODEL -- data -- src ---- folder a ---- folder b ---- driver ------- main.py ------- config.py ------- file ------- file

__main__.py

contains some of the code, but calls upon the other folders when doing the computations, etc. the data folder is where I need to call the API data into, and where some data will be extracted from after the model has ran. I hope this helps!

Can you show a simple example of what your

__main____.py

looks like?