Second question - we use django's ORM for db models. Are there any issues with passing these models between tasks? I think they're serializable.

i can't say im so familiar with django internals but even if they're not serializable you can still pass them around, you can exclude problematic things https://docs.prefect.io/v3/advanced/caching#handling-non-serializable-objects

Depends on how big they are -- the prefect behavior when the args serialize too big has been problematic for me https://github.com/PrefectHQ/prefect/issues/18546

we should definitely try to raise a more informative error here if we can but ill say the refrain which is that if you're even approaching the limit you're probably passing a lot more over the wire than you want, and should instead pass a reference to the data and load it from the flow

Yeah, I fixed mine, but OP wants all his tasks to be separate processes which means they might always have to go over the wire?

oh i didn't pick that up from OPs message but if someone hits some weirdness im happy to take a look

I don't think I need them to be separate processes - my understanding was that each task is a thread, not a process?

yea by default if you use

.submit

its threads, but you can choose a

ProcessPoolTaskRunner

if you want process isolation (but yea you generally don't need that if you don't know you need it)

Yeah, I think we'll just stick with the default task runner - thanks for the clarification!