Netstat analysis for outgoing/incoming connections

netstat supports a set of options to display active or passive sockets. The options –t, –u, –w, and –x show active TCP, UDP, RAW, or Unix socket connections. If you provide the –a flag in addition, sockets that are waiting for a connection (i.e., listening) are displayed as well. This display will give you a list of all servers that are currently running on your system.

Invoking netstat -ta on vlager produces this output:

$ netstat -ta (you can also use -n option to avoid the dns lookup for displaying domain names, which will save performace).

Active Internet Connections
Proto Recv-Q Send-Q Local Address Foreign Address (State)
tcp 0 0 *:domain *:* LISTEN
tcp 0 0 *:time *:* LISTEN
tcp 0 0 *:smtp *:* LISTEN
tcp 0 0 vlager:smtp vstout:1040 ESTABLISHED
tcp 0 0 *:telnet *:* LISTEN
tcp 0 0 localhost:1046 vbardolino:telnet ESTABLISHED
tcp 0 0 *:chargen *:* LISTEN
tcp 0 0 *:daytime *:* LISTEN
tcp 0 0 *:discard *:* LISTEN
tcp 0 0 *:echo *:* LISTEN
tcp 0 0 *:shell *:* LISTEN
tcp 0 0 *:login *:* LISTEN

This output shows most servers simply waiting for an incoming connection. However, the fourth line shows an incoming SMTP connection from vstout, and the sixth line tells you there is an outgoing telnet connection to vbardolino.[1]

Using the –a flag by itself will display all sockets from all families.

Notes
[1]

You can tell whether a connection is outgoing from the port numbers. The port number shown for the calling host will always be a simple integer. On the host being called, a well-known service port will be in use for which netstat uses the symbolic name such as smtp, found in /etc/services.

Using vmstat for virtutal memory analyze

vmstat, as its name suggests, reports virtual memory statistics. It shows how much virtual memory there is, how much is free and paging activity. Most important, you can observe page-ins and page-outs as they happen. This is extremely useful.

To monitor the virtual memory activity on your system, it's best to use vmstat with a delay. A delay is the number of seconds between updates. If you don't supply a delay, vmstat reports the averages since the last boot and quit. Five seconds is the recommended delay interval.

To run vmstat with a five-second delay, type:

vmstat 5

You also can specify a count, which indicates how many updates you want to see before vmstat quits. If you don't specify a count, the count defaults to infinity, but you can stop output with Ctrl-C.

To run vmstat with ten updates, five seconds apart, type:

vmstat 5 10

Here's an example of a system free of paging activity:

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
0 0 0 29232 116972 4524 244900 0 0 0 0 0 0 0 0 0
0 0 0 29232 116972 4524 244900 0 0 0 0 2560 6 0 1 99
0 0 0 29232 116972 4524 244900 0 0 0 0 2574 10 0 2 98

All fields are explained in the vmstat man page, but the most important columns for this article are free, si and so. The free column shows the amount of free memory, si shows page-ins and so shows page-outs. In this example, the so column is zero consistently, indicating there are no page-outs.

The abbreviations so and si are used instead of the more accurate po and pi for historical reasons.

Here's an example of a system with paging activity:

procs memory swap io system --- cpu ---
r b w swpd free buff cache si so bi bo in cs us sy id
. . .
1 0 0 13344 1444 1308 19692 0 168 129 42 1505 713 20 11 69
1 0 0 13856 1640 1308 18524 64 516 379 129 4341 646 24 34 42
3 0 0 13856 1084 1308 18316 56 64 14 0 320 1022 84 9 8

Notice the nonzero so values indicating there is not enough physical memory and the kernel is paging out. You can use top and ps to identify the processes that are using the most memory.

You also can use top to show memory and swap statistics. Here is an example of the uppermost portion of a typical top report:

14:23:19 up 348 days, 3:02, 1 user, load average: 0.00, 0.00, 0.00
55 processes: 54 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 0.0% user, 2.4% system, 0.0% nice, 97.6% idle
Mem: 481076K total, 367508K used, 113568K free, 4712K buffers
Swap: 1004052K total, 29852K used, 974200K free, 244396K cached

Sending mail through command line client.

Some days back I'm in situation to test the continuous spamming of mails to my mail box.

This command did me the work

for i in `seq 1 35`; do echo "test" | mail -s test forum@email.com; done

Setup YUM installer repo

To use the YUM installer you need to register the repo.

Create a file "/etc/yum.repos.d/centos.repo" and paste the below content.

[base]
name=CentOS-$releasever - Base
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
baseurl=http://mirror.centos.org/centos/5/os/i386/
gpgcheck=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-centos4
protect=1

AWK to take last previous column

You can use the below command to print last previous column.

awk '{print $(NF-1)}' access.log | sort | uniq | wc -l

Recursive shell command s

I was searching for macking the commands work recursively.

Below is the way to do it.

MIGRATION_CONFIG_FILE="$(dirname $(dirname $(readlink $0)))"/$CONFIGURATION_FILE.

Shell script to connect postgress in a loop

i=211
while [ $i -le 250 ]
do
folder="psr-pub-$i"
echo "Starting processing $folder."
./svnmapping.sh $folder
echo "Done with $folder"
i=`expr $i + 1`
done


projectName=$1
userId="user1308"

DBConnect="psql -d psrdb -U psruser --tuples-only --single-line"

date=`date +"%F %T"`
projId=`echo "select id from project where path='projects.$projectName'" | $DBConnect`
projId=`echo $projId`

parent_folder_id=`echo "select id as parent_folderid from folder where path = 'scm' and project_id='$projId'" | $DBConnect`
parent_folder_id=`echo $parent_folder_id`

nextscm_id_query="select max(substring(id from (char_length('reps')+1))) from scm_repository"
primaryId_query="select id from objecttype where application='Scm' and name='Repository'"
secondaryId_query="select id from objecttype where application='Scm' and name='Commit'"

nextscm_id=`echo $nextscm_id_query | $DBConnect`
nextscmId=`echo $nextscm_id`
nextscmId=reps`expr $nextscmId + 1`
primaryId=`echo $primaryId_query | $DBConnect`
primaryId=`echo $primaryId`
secondaryId=`echo $secondaryId_query | $DBConnect`
secondaryId=`echo $secondaryId`

insertscm="insert into scm_repository values('$nextscmId', 'exsy1002', '/svnroot/$projectName', 'f')"
insertfolder="insert into folder values ('$nextscmId', 'scm.$projectName', '$projectName', '$projectName', '100', '$date+05:30', '$date+05:30','f', '$primaryId', '$secondaryId', '$userId', '$userId', '$parent_folder_id', '$projId', '0')"

scm=`echo $insertscm | $DBConnect`
fol=`echo $insertfolder | $DBConnect`

echo $scm" "$fol

AWK IF ELSE block

nawk -F"," ' BEGIN{OFS=","} {if (NR==1) {
print "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx"
} else {
print $1,$2,$3,$4,"S",1
print $1,$2,$3,$5,"S",2
print $1,$2,$3,$6,"D",1
print $1,$2,$3,$7,"D",2
}}' in.txt

Extract total number of requests hits

Below is the command to calculate no of time projects accessed using the url. The informations are retrieved from log file where all the requests are logged.

cut -d ' ' -f 7 ssl_request_log |grep '/sf/'|cut -d '/' -f 3|sort|uniq -c

cut -d ' ' -f 7 ssl_request_log |grep '/sf/'|sort|uniq -c

Sample log file

[31/Aug/2008:05:50:08 +0200] 0.0.0.0 SSLv3 RC4-MD5 "GET /asf-help/RoboHelp_CSH.js HTTP/1.1" 7677
[31/Aug/2008:05:50:08 +0200] 0.0.0.0 SSLv3 RC4-MD5 "GET /wsf-images/masthead/logo.gif HTTP/1.1" 2430
[31/Aug/2008:05:50:09 +0200] 0.0.0.0 SSLv3 RC4-MD5 "GET /fsf-images/masthead/dropdown.gif HTTP/1.1" 49
[31/Aug/2008:05:50:09 +0200] 0.0.0.0 SSLv3 RC4-MD5 "GET /nsf-images/masthead/help.gif HTTP/1.1" 402

AWK Samples

Here are some of the useful awk program collections!

awk '{print $2,$1}' filename
awk '{print $2,$1}' filename
awk '$1 > $2 {print $1,$2,$1-$2}' filename

awk '$1 > $2'           filename
awk '$1 > $2{print}'    filename
awk '$1 > $2{print $0}' filename

awk '$1=="foo"{print $2}' filename
awk '/foo.*bar/{print $1,$3}' filename
awk '$2~/foo/{print $3,$1}' filename
awk '$2!~/foo/{print $3,$1}' filename

awk '/foo/,/bar/' filename

awk 'BEGIN{print"fee"} $1=="foo"{print"fi"}
  END{print"fo fum"}' filename

awk '{print $1,$NF }' filename
awk '{print NR,$0 }' filename
awk -F: '{print $1,$3 }' /etc/passwd
awk '{$10=""; print }' filename
awk '{for(i=1;i<=NF;i++) print $i }' filename awk '{printf("%s %03d %02d %.15g\n",$1,$2,$3,$3/$2); }' filename
awk '{for(i=NF;i > 0;i--) printf("%s",$i); printf("\n"); }' filename

Replace particular field of line using awk

I was trying to replace the particular field of a line and found a suggestion for that.

echo 'count ****** 50' | gawk '{$2="\"5AB0\"";print}'

The above gawk command replaces the ****** with 5ABo.

Create master and slave subversion repository setup using svnsync

I've learned the one more svn command svnsync, which I ever tried before. thought sharing this will be usefull for svn buddies.

Basically this command is to replicate the svn repository (i.e) Master to n number of slave repository setup.

How to:

1. Create the slave repository. follow the below steps for creating that.

a. Create an new repository.
b. Then you need to alter somescripts in the newly created repository inorder to accept this setup.
c. Get in to hooks directory of newly created above repository and rename pre-revprop-change.tmpl to pre-revprop-change.
d. Give pre-revprop-change executable permission.
e. Then edit the pre-revprop-change and add exit 0 at the begining of the file.

2. Execute the below commands to point master repository to newly created slave

1. svnsync init slaverepositoryurl masterrepositoryurl
The above command is one time setup command (i.e) pointing the master repository to slave repository.
2. svnsync sync slaverepositoryurl
The above command is invoked when ever you need to sync with master repository.

Similarly you can have n number of slaves for the master repository.

Also heared that this is not the right way to use it in production, but any how Im posting this to get startup.

Making your vi for default settings

If you want your vi editor to load with default settings (i.e) the line number to be displayed and also tabspace to set for 4 characters.

Here the way to do...

create a file ~/.exrc with settings

set nu
set tabstop=3

Then opening the vi will have these settings common for all.

SSH key generation

We will be ended up by re entering the password when ever we connect to a remote machine using ssh.

Here is the way to avoid re entering the password each and every time by generating a key pair Id and moving to remote machine only once.

ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/localuser/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/localuser/.ssh/id_dsa.
Your public key has been saved in /home/localuser/.ssh/id_dsa.pub.
The key fingerprint is:
93:58:20:56:72:d7:bd:14:86:9f:42:aa:82:3d:f8:e5 localuser@mybox.home.com

scp ~/.ssh/id_dsa.pub username@remotemachine:.ssh/authorized_keys


login to remote machine

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Power of Shell Script

Scripts makes the job more easy !

Below is the sample script which I wrote to retrieve the count of files modified for all the modified dates in home directory. This script also gives the sum of bytes for files modified on the corresponding dates.

prakash@prakash-laptop:~$ for filedate in `ls -l | awk '{print $6}' | sort | uniq -c | awk '{print $2}'`;do ls -l | grep $filedate | awk 'BEGIN{print "----------------------------"} {count+=1;sum+=$5} END{print $6" | " count " | " sum }' ; done


Sample Output
----------------------------
Date | Count | Sum of bytes
----------------------------
2007-10-30 | 1 | 26
----------------------------
2007-11-21 | 1 | 4096
----------------------------
2008-02-06 | 1 | 332341
----------------------------
2008-02-25 | 2 | 8192
----------------------------
2008-03-03 | 1 | 999
----------------------------
2008-03-04 | 1 | 219
----------------------------
2008-03-05 | 1 | 3402
----------------------------
2008-03-06 | 2 | 739209
----------------------------
2008-03-11 | 4 | 6648
----------------------------
2008-03-14 | 10 | 50918
----------------------------
2008-03-15 | 4 | 106573
----------------------------
2008-03-16 | 1 | 10635


Really the scripting makes your job more easy.

Ignoring Unix signals

Here is the interesting small C program I came across to ignore the unix signals!. Basically I'm a Java programmer, so this small C impressed me a lot :)

Below program ignores the CTRL + C signal,

test.c

#include < stdio.h >
#include < signal.h >


main()
{
signal(SIGINT,SIG_IGN);
while(1)
printf("You can't kill me with SIGINT anymore, dude\n");
return 0;
}


gcc test.c -o testout
./testout

Running the above program completely ignores the CRTL + C command and keep on running with out aborting. CRTL + C sends the SIGINT signal and we ignore that in program.

UNIX SIGNALS

THE UNIX SIGNALS are used for communicating the process when ever the event has occurred.

When ever you give CRTL + C on the running process or kill on the running process id, the signals are sent to process to notify the event.

There are about 64 signals and can be viewed using kill -l.

When you press CRTL + C , by default SIGINT is sent to the running process,
When you press CRTL + Z , by default SINTSTP is sent to the running process,
When you press CRTL + / , by default SIGABRT is sent to the running process.

Think when some one press CTRL + C when your program is half the way, you are going to end up with non cleaned up resource, this is because you do'nt have handler over the SIGINT in our program.

You can write the handler for CRTL + C using trap as follows to get the control. You cannot get the handler for SIGKILL and SIGSTOP. These two signals cannot be caught while other can be.

test.sh

!/bin/bash

trap "echo 'This is trap executing for 0!'; exit 0" 0
trap "echo 'This is trap executing for SIGINT!'; exit 1" SIGINT
trap "echo 'This is trap executing for SIGKILL!'; exit 1" SIGKILL
trap "echo 'This is trap executing for SIGTERM!'; exit 1" SIGTERM
trap "echo 'This is trap executing for SIGABRT!'; exit 1" SIGABRT

echo "We are running the script. Press Ctrl-C to cause trap to execute!"

read

We can also send the signals during kill using pid.

For EX:
kill -15 pid,

The above command will send the SIGTERM signal to the pid before terminating. So you can do your clean up logic by handling the signal in your program.

Using kill -9 pid,

the above command will send the SIGKILL signal but it cannot be caught by the program, this is also similar to SIGSTOP that cannot be caught too.

So it's advisable to use the SIGTERM to kill the process and then the SIGKILL as we can caught the SIGTERM signal for our operations and the SIGKILL cannot be.

The Recommended order of killing the process id is

kill_pid () {
PID=$1
RETVAL=0

for signal in "TERM" "INT" "HUP" "KILL"; do
kill -$signal $PID
RETVAL=$?
[ $RETVAL -eq 0 ] && break
echo "warning: kill failed: pid=$PID, signal=$signal" >&2
sleep 1
done

return $RETVAL
}

kill_pid 1234

How to find processor 32 bit or 64 bit

Some time you may need to know what bit your processor uses, for installing the packages.

Here is the way you can use to find the information.

getconf LONG_BIT

uname -m

(x86_64 GNU/Linux indicates you have 64bit CPU. If you use see i386/i486/i586/i686 then it is a 32 bit CPU.)




To find opened ports in linux

After a long search I found the good way to search the opened ports in a machine.
It's nmap & netstat.

Syntax

netstat -na | grep 6000

nmap -p 1-63335 localhost

The above command displays the ports opened in machine localhost.

Mastering VI

D	To delete remaining characters under cursor in current line.
dd	To delete entire line and copy to clipboard
$	Move the cursor to the end of the current line.
C-g	To display filename and number of lines in end of the terminal
r	To replace single character under cursor
R	To replace multiple character under cursor untill escape is pressed
dw	To replace a word under cursor