Monthly Archives: February 2017

Partition-Aware Data Loading in Spark SQL

Knoldus Blogs Data loading, in Spark SQL, means loading data in memory/cache of Spark worker nodes. For which¬†we use to write following code: val connectionProperties = new Properties() connectionProperties.put(“user”, “username”) connectionProperties.put(“password”, “password”) val jdbcDF = spark.read .jdbc(“jdbc:postgresql:dbserver”, “schema.table”, connectionProperties) In … Continue reading

Posted in Scala | Leave a comment