hey prefect community. i am new on a team that's using prefect. my mandate is to make sure we're making the most of the framework. our codebase is almost entirely OOP, and we wrap all of our flow logic in just one task for each flow. i dont think it would be wise for us to shift the paradigm we're programming in completely, but i do want us to leverage the task decorator and observability we get by writing functional code. 1. why does prefect force functional > OOP? Is this the norm for ETL, outside of prefect as well? 2. what benefits do we gain by going functional? 3. when is OOP appropriate? our extract logic is a web scraping so we have a lot of helper functions that share the web driver. also, a web driver instance isn't serializable. 4. what else should i be thinking about here?

I don’t have time to dig into all of this so hopefully someone else can respond too 🙂 but in brief, we have a focus on functional usage because we allow you to very easily run your code concurrently and distributed. When mutating a shared object in these execution patterns, you’re pretty likely to encounter behavior that is confusing. In contrast, a functional style scales well with concurrency because each task has well defined inputs and outputs — they’re less likely to rely on shared state.

1

Jon, Michael's response is perfect, I don't have much to add - tasks/flows are supposed to be stateless because each run might be running on an entirely different container/pod/infra, might need to be restarted, retried, and it's recommended to leverage functional programming style to pass data when needed without any shared global state

We have many customers using classes to define their tasks in Prefect 1 — the caveats I mentioned still apply though.

Why force everything OOP?

thank you so much @Zanie and @Anna Geller for the thoughtful responses