Does anyone know how to access task results but not have them persisted to disk? I'm getting a

No space left of device

error when Prefect is calling

await PersistedResult.create

, but if I turn

persist_result=False

in the task I cant actually use it (as in its not available in memory).

How big are the results? Prefect is trying to persist them as json with base64-string-encoding. Maybe look into whether they could be persisted more efficiently in your own data store (as binary) and then the only persist the "key" to that data store as the result of the task.

Gigabyte size dataframes. I just want to pass them from one task to another with no persistence

Look at the "quote" function here https://docs.prefect.io/v3/develop/big-data#remove-task-introspection

Ah I didnt realise quote might stop persistence, cheers, I'll check it out, thanks mate 🙂