Luis Muniz
09/02/2020, 1:40 PMjosh
09/02/2020, 2:20 PM@resource_manager(name="spark")
class SparkCluster:
def __init__(self, ...):
self.... = ...
def setup(self):
return SparkClient(self...)
def cleanup(self, client):
client.close()
SparkClient
would be whichever library you use for managing EMR connectionsLuis Muniz
09/02/2020, 3:03 PM