General Q: What's the best way of dealing with flows with python package dependencies? For example, I use PySFTP, PyODBC, SqlAlchemy and pandas. So when I deploy this and it runs on an azure vm with none of these dependencies installed.. how best to deal with it in terms of... which storage? which agent? Do I package it up as a docker image as per @Kevin Kho’s excellent article here: https://medium.com/the-prefect-blog/the-simple-guide-to-productionizing-data-workflows-with-docker-31a5aae67c0a Looking for general advice / see what you guys do

@Adam Everington packaging your dependencies into a Docker image is a great idea! The article above shows how you can do that. In general, the agents match with run configurations e.g. DockerRun with Docker agent, KubernetesRun with Kubernetes agent, etc. And when it comes to storage, you can either package your flow together with your dependencies using Docker storage: https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows/docker_pickle_kubernetes_run_custom_ecr_image.py Or you can choose to package your dependencies separately from the flow storage - this would mean that you reference your custom image with dependencies on the run configuration, and for the flow storage you may leverage one of Git storage classes (e.g. GitHub storage) or one of cloud storage classes (e.g. S3). Here is an example that shows this pattern: https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows/github_kubernetes_run_custom_ecr_image.py Here you can find more examples incl. an example Dockerfile and commands to build a custom image https://github.com/anna-geller/packaging-prefect-flows

@Anna Geller your github is becoming a little bible for me, thanks for your help & contributions

correct, I’m working from Berlin - Germany 🙂

Germany is absolutely beautiful!

And you’re from UK?

I am, the less said about that the better...