Snowflake Connector for Spark

The Snowflake Connector for Spark (“Spark connector”) brings Snowflake into the Apache Spark ecosystem, enabling Spark to read data from, and write data to, Snowflake. From Spark’s perspective, Snowflake looks similar to other Spark data sources (PostgreSQL, HDFS, S3, etc.).

Snowflake supports three versions of Spark: Spark 2.2, Spark 2.3, and Spark 2.4. There is a separate version of the Snowflake Spark Connector for each version of Spark. Use the correct version of the connector for your version of Spark.

The connector runs as a Spark plugin and is provided as a Spark package (spark-snowflake).

Version 2.4.0 (and higher) of the Spark connector support Snowflake on both Microsoft Azure and AWS. Earlier versions support Snowflake only on AWS.

Note

Currently, the Spark connector does not work with Snowflake accounts hosted on Google Cloud Platform.

Next Topics:

• Overview of the Spark Connector
• Installing and Configuring the Spark Connector
• Configuring Snowflake for Spark in Databricks
• Configuring Snowflake for Spark in Qubole
• Using the Spark Connector