SQOOP  : mapred.FileAlreadyExistsException : Output directory
Sometimes when you import data from RDBMS to Hadoop via Sqoop you will see this error. org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://hadoopcluster/user/username/importtable  already exists Solution: $ hdfs dfs -rm -r -skipTrash  hdfs://hadoopcluster/user/username/importtable Reason: When Sqoop is used for Importing data, sqoop creates a temporary file under  home directory and later deletes those files. Sometimes due to some issue,… (0 comment)

gzip / bzip2 can compress individual files. To compress entire folder structure ‘tar’ command comes to our rescue. Compress to .gz  tar -zcf </path/output.gz>  /sourcefolder/path Compress to .bz2 tar -jcf </path/output.bz2> /sourcefolder/path tar options : -z : gzip -c : compress -f : filename -j : bzip2  … (0 comment)

DECLARE @LastmonLastBusinessDay date, @inputdate date SET @inputdate = ’05/25/2014′ SELECT @LastmonLastBusinessDay = DATEADD(ms,-3,DATEADD(mm,DATEDIFF(mm,0,@inputdate),0)) – CASE DATENAME(dw,DATEADD(ms,-3,DATEADD(mm,DATEDIFF(mm,0,@inputdate),0))) WHEN ‘SUNDAY’ THEN 2 WHEN ‘SATURDAY’ THEN 1 ELSE 0 END print @LastmonLastBusinessDay… (0 comment)

Recently I have been noticing some weird special characters on my PuTTy screen.  There was no pattern to it. It will appear all of a sudden. Its so annoying when you are working on a script. ex: ^[[28~ Reason : To keep my windows active from locking, I had installed an application called ‘caffeine’ .  Caffeine… (0 comment)

Here is a simple command line statement to kill frozen windows task manager. START – RUN – CMD (make sure you are running it as Administrator) C:>taskkill /F /FI “WindowTitle eq Windows*” Taskkill – Command /F – Forcefully terminate the process(es) /FI – Filter “WindowsTitle” – valid filter eq – valid operator Note: C:> taskkill… (0 comment)