Script(s)

technology as-is

August 28, 2015
by gchandra
0 comments

Hive : Create database in different folder

When database is created in Hive, its usually saved in default folder  /user/hive/warehouse/yourdabase.db

If alternate location is needed, then

create database <yourdatabase> location  ‘/db/path/yourdatabase.db’    [single quotes]

 

August 28, 2015
by gchandra
0 comments

Linux : Compress entire folder to .gz or .bz2

gzip / bzip2 can compress individual files. To compress entire folder structure ‘tar’ command comes to our rescue.

Compress to .gz 

tar -zcf </path/output.gz>  /sourcefolder/path

Compress to .bz2

tar -jcf </path/output.bz2> /sourcefolder/path

tar options :

-z : gzip
-c : compress
-f : filename
-j : bzip2

 

August 17, 2015
by gchandra
0 comments

Error : bash: 08: value too great for base (error token is “08”)

Error Message : bash: 08: value too great for base (error token is “08”)  or  bash: 09: value too great for base (error token is “09”)

This error message is seen when an arithmetic operation is performed on numbers which has value 08 or 09.  Mostly when dealing with months and days.

Reason : As the number start with 0, linux assumes it as number with Octal base and since there are no 8 and 9 in octal it throws the above error.

Solution : Convert the number 08 to 8, do the arithmetic operation and later convert it back if needed.

Example :

$ mm=08

#Converting  08 to 8
$ mm1=$(echo $mm| awk ‘{printf “%d\n”,$0;}’)
$ echo $mm1

# Adding 1 to 8
$mm1=$(($mm1+1))
$ echo $mm1

# Converting 9 to 09
$mm2=$(printf %02d $mm1)

$echo $mm2
09

August 17, 2015
by gchandra
0 comments

Linux Tip : Convert date time to Unix Time (Epoch)

Simple script to covert  regular  date time to unix epoch time format  (UTC)

$dat=’01/02/2015′

#If date is passed without time then its assumed as midnight  00:00:00 for that day.

$sepoch=$(date “+%s” -d “$dat”)

$echo $sepoch

will return 1420156800

#If you need for specific datetime, then add time (24 h format)

$sepoch=$(date “+%s” -d “$dat  13:15:00”)

$echo $sepoch

will return 1420204500

This can be validated using  www.epochconverter.com

August 17, 2015
by gchandra
0 comments

Linux Tip : Split Date into Day, Month, Year

Easy way to split date into  Day, Month, Year  in Linux.  This code was tested using Bash shell.

$dat=’13/2/2015′

sday=$(date -d “$dat” ‘+%d’)
smonth=$(date -d “$dat” ‘+%m’)
sfourdigityear=$(date -d “$dat” ‘+%Y’)
stwodigityear=$(date -d “$dat” ‘+%y’)

echo $sday  will display   02
echo $smonth  will display   13
echo $sfourdigityear  will display   2015
echo $stwodigityear  will display   15

 

August 17, 2015
by gchandra
0 comments

Hive Error : SemanticException TOK_ALLCOLREF

Hive Error : 

SemanticException TOK_ALLCOLREF is not supported in current context

Reason : 

Select Distinct * From <tablename>

Solution :

Hive doesn’t support  distinct *, so  mention the column names

Select Distinct col1, col2, col3… From <tablename>