This script loops through hdfs files system and reads the first line and writes it to console.  Most part its self explanatory. This script uses pipeline delimiter  “|” .  Its optional and can be skipped. import org.apache.hadoop.fs.Path import org.apache.hadoop.conf.Configuration import org.apache.hadoop.fs.FileSystem   val path = "/hdfspath/" val conf = new Configuration() val fs = FileSystem.get(conf)… (0 comment)

Its pretty straight forward to add Command Line Arguments to Spark (scala) from shell. $ ./spark-2.0.0-bin-hadoop2.6/bin/spark-shell -i ~/scalaparam.scala --conf spark.driver.args="param1value  param2value  param3value" Parameter values are separated by  spaces  (param1value  param2value  param3value) contents of  scalaparam.scala val args = sc.getConf.get("spark.driver.args").split("\\s+") val param1=args(0) val param2=args(1) val param3=args(2) println("param1 passed from shell : " + param1) println("param2 passed from… (0 comment)