Ken Nguyen
03/23/2022, 11:21 PMwith Flow("flow", run_config=RUN_CONFIG, storage=STORAGE) as flow:
json_data = get_json_data(
url, query, headers,
task_args={"name": "Getting Flow Data"}
)
How can I then access json_data as a python object, rather than a FunctionTask object?Kevin Kho
03/23/2022, 11:22 PMKen Nguyen
03/23/2022, 11:25 PMfirst_row = json_data[0,]
2. pass dataframe into the next task
a. processed_df = process_dataframe(json_data, task_args={"name": "Processing df"})
Kevin Kho
03/23/2022, 11:31 PMjson_data
is of type Task until execution time of the flow. The json_data[0,]
syntax accesses the Task because it runs during build time, not run time. So you need to put this logic inside a Task so that it will defer to run time instead of build time. Does that make sense?Ken Nguyen
03/23/2022, 11:34 PM<http://logger.info|logger.info>(json_data)
However, I don’t actually see anything get printed in the logs. Could you tell me why that’s happening?Kevin Kho
03/23/2022, 11:58 PM