Thread
#prefect-community
    Michael Wooley

    Michael Wooley

    1 year ago
    Hello, my question became somewhat lengthy so I've placed it in the attached snippet. The tl;dr is: Can I
    yield
    part of a task result for use downstream? If not, what is the best alternative? Or am I just thinking about this all wrong?
    Michael Adkins

    Michael Adkins

    1 year ago
    Hi @Michael Wooley -- you can't yield data from a task right now (and I do not think Dask will make it easy to implement). I think your best option is writing the data to files then passing a reference to the files to your mapped transform as you mentioned. If you pass the data itself instead of a pointer then you will almost certainly run into memory constraints.
    Michael Wooley

    Michael Wooley

    1 year ago
    Okay, just wanted to confirm. Thanks @Michael Adkins!