Good morning, Prefectionists! We are sailing. I have successfully deployed two very simple Workflows about a month ago and everything is working beautifully. I'm a DBA for a small non-profit and we handle a ton of data. We decided to implement prefect so I can automate many of the data importing and ETL tasks I perform manually. I wanted to share a couple of things Prefect taught me. 1:Register the flow on the agent its intended to run on. for some reason this resolved all the import statements correctly. 2:Translating your workflow into prefect code helps to refactor the steps of the flow. I would do these in SSIS, and SSIS made the "theory" of the flow easy, but it didn't truly represent the logic of the flow. In doing so, I developed a much more concise workflow, which had a few more tasks. Let this happen: its a good thing. 3: Python is amazing. And since Prefect doesn't get in the way of that, prefect is amazing. It quite literally can do anything. 4: Docker flows: This can get messy. I found about 52 containers that were left over from all my flows being run. This is a really intelligent solution regardless, it just leave a trail. I'm sure there is a setting to automatically remove these that I just haven't found. Don't be alarmed if you have clutter in your docker instance. It well worth it. 5: When deciding to implement a flow in Prefect, you are forced to design the whole thing. No more "Oh, we'll deal with that when we get there..." This is a steep hill to climb, but you once you do, its a never-ending coast down the other side. This forces questions to the beginning of the process and helps get solid answers for them right away. Its possible that a workflow may not be a good candidate for Prefect, or maybe parts of the workflows are candidates. You actually save a lot of time designing end to end. Thanks for reading.

We usually call Prefect users “Prefectionists”! 😄 Thank you so much for sharing your experience! 👏 #1 - nice that your imports work as intended - if you face any issues with packaging your code dependencies, let us know, we have some resources that can help with that #2 - I definitely empathize with you since I also used SSIS, Pentaho Data Integrations and similar drag-and-drop ETL madness and I can attest that it’s not a good experience, creates a ton of tech debt, is hard to scale and makes it really hard to implement best engineering practices #3 That’s true! 💪 #4 - You could use a ShellTask in a flow running e.g. every day/week and running the command

docker container prune

This allows you to remove stopped containers and reclaim the disk space. You may even do:

docker system prune

#5 - I agree, Prefect allows you to build workflows in a very modular and flexible way. Again, thanks for sharing! 👏

standing ovation

Thanks, Anna. I will definitely use your suggestions.