Emilio Guzman

08/29/2023, 3:28 AM
Hello, I want to create a spark session and use a GCS connector via this Hadoop configuration: spark = SparkSession.builder \ .appName(‘test’) \ .config(“”, path_to_credentials) \ .config(“”, “”) \ .config(“”, “”) \ .config(“spark.jars”, “/Users/og/Downloads/gcs/gcs-connector-hadoop2-2.2.15-shaded.jar”) \ .getOrCreate() Is there any method I can use to use a prefect google cloud block to paste the path to my credentials? or is there any other method ? Any advice helpful! Thank you.