This may be (or rather, is) an entirely “opinion” question: I often have the situation where I need a modern substitution for a Makefile. Don’t get me wrong, the old-school GNU Make is great at what it does, but the syntax can be, ahem, the polite word is “painful”. I would be curious to gathering some ideas: Prefect has something akin to dependency management, but there are other Python-based tools for this as well: Invoke, for example. Note that these are strictly limited to problems that will never need any sort of massive orchestration, so I would be curious what the community thinks: can Prefect solve the Makefile problem as well, or would that be bending a tool to do a task it is not designed for? And, have a nice start to the week!

This is something we are actively looking into for the near future. Specifically, how to make it effortless for users to package dependencies together. Is that your use case for the Makefile? or are you trying to run other commands and you’re suggesting a Prefect Flow run those commands?

To dependency packaging: it would be in that case be very nice to have some sort of way (or, if it exists, an example in the docs) to “publish” a (collection of) flow(s). So, on the machine running the flows, I just do

pip install <blah blah>

My use case described above is a bit different though, and probably fits more into the CI world: 1) build your program 2) run tests with the build 3) if the build works, publish 4) publish docs (a very simplified example, 3+4 can happen at the same time) . It slowly in that way goes back to DAGs which I think was one of the things you were trying to partially get away from, and I am sure you can bend prefect to do this as well. Overall though, I’ve been super impressed. given how much weird hacks I have had to do with other systems, I somehow have learned to view prefect as the “all purpose tool” :-)

Yes we have had users use Prefect as the backbone of some sort of CI/CD. Changes to source trigger a Flow that does the tests and publishes. Thanks for that!

No problem, always happy to give user end feedback 🙂 So, are there recommendations for how best to package up flows for reuse and distribution? That would good to read through at some point this evening (I still need to write the flows first though, hehe)

For Prefect 1.0, plain Docker. But for 2.0, we’re looking into how to make the experience easier