Jason Prado

05/05/2021, 5:28 PM
I have a few Airtable tasks I’ve written and reused a few times. They snarf a whole Airtable table into a DataFrame, apply updates, and insert new records. The existing Airtable tasks in prefect only insert/read records a row at a time. I’d be interested in contributing them. Here’s the source: I would guess that most people want to read records from Airtable into python dicts/lists rather than DataFrames, right? If I wrote these classes that way but still wanted to use DFs in my project, would I then write my own tasks that turn the dicts/lists into DataFrames? Or should the tasks take a parameter to specify what kind of output they want (dicts vs DFs)? Just curious what the most idiomatic way to publish these tasks would be.
👀 1

Kevin Kho

05/05/2021, 6:08 PM
Hi @Jason Prado! I think returning base Python objects is better because users may be using different kinda of dataframe (Spark, Pandas, Arrow) so it makes sense to return base Python to that they can instantiate the data frame on their end? Just my thoughts.
👍 2