Spark sql with JDBC
./sbin/start-master.sh you will get master is running on a hostname and a port number. copy that and put in blow command ./bin/spark-shell --driver-memory 4G --master spark://master-host:7077 --executor-memory 8G --executor-cores 1 --num-executors 3 import java.util.Properties val connectionProperties = new Properties() connectionProperties.put("user", "actualUsername") connectionProperties.put("password", "actualPassword") val jdbcUrl = "jdbc:mysql://hostname/dbname" val sqlquery = "(select * from t1 limit 10)tmp" val df = spark.read.jdbc(url=jdbcUrl, table=sqlquery, properties=connectionProperties) df.show for ms sql: https://docs.microsoft.com/en-us/sql/connect/jdbc/building-the-connection-url?view=sql-server-2017