Iterate files in folder using Spark Scala
This script loops through hdfs files system and reads the first line and writes it to console. Most part its
Read moreSpark
Spark Scala Command line Arguments
Its pretty straight forward to add Command Line Arguments to Spark (scala) from shell. $ ./spark-2.0.0-bin-hadoop2.6/bin/spark-shell -i ~/scalaparam.scala –conf spark.driver.args=”param1value
Read more