Iterate files in folder using Spark Scala
This script loops through hdfs files system and reads the first line and writes it to console. Most part its
Read moreThis script loops through hdfs files system and reads the first line and writes it to console. Most part its
Read moreIts pretty straight forward to add Command Line Arguments to Spark (scala) from shell. $ ./spark-2.0.0-bin-hadoop2.6/bin/spark-shell -i ~/scalaparam.scala –conf spark.driver.args=”param1value
Read more