I'm sure I'm doing something wrong here, but I get the oddest message when I try to schedule a flow that I registered:

Unexpected error: TypeError("cannot pickle '_thread.lock' object")

Can someone let me know what this is if they've had this same issue? My flow runs ok from the machine where I have the agent, but I can't get it to run when I attempt scheduling or even doing an ad-hoc run from the prefect cloud UI

Hi Jacob, are you using threading locks in your flows/tasks?

(or something that uses a thread lock internally, like a database connection)

Data passed between tasks for example needs to be pickleable https://docs.prefect.io/core/concepts/common-pitfalls.html#task-results-and-inputs

I'm using Snowflake - could that be it?

Jake sent the doc 🙂 but yeah you generally can't pass database connections across tasks.

Oops I must not have hit enter when I made the other mention. Thanks @Jake Kaplan!

@Jacob Bedard just to clarify: xcoms store your data in a database (or other external backend), and then retrieve when you pull it. When you pass data in Prefect, it’s passed in memory. But when you use Dask executor, then those return values need to be passed between Dask workers, and this is why your task return data has to be serializable with cloudpickle.

I've taken some time to read up on that since first asking about it. I think I have a clear idea of what I can and can't pass between tasks now. Thanks for confirming! BTW I've got my flow in production now. I love the GUI for schedule-building. Some things are better done as code, but the GUI is just perfect for this job in particular, especially after a few days of setting up resources and learning how to use this new framework. I was so relieved to just see this nifty little scheduler and the automations UI. Just awesome.

thank you so much @Jacob Bedard, I’ll pass it to the UI team