1. Replace “www.trendics.com” with “tools.trendics.com” in all html files in the current directory…

find . -maxdepth 1 -name "*.html" -type f -exec sed -i 's/www.trendics.com/tools.trendics.com/' {} \;

2. Replace “www.trendics.com” with “tools.trendics.com” in all text files and all subdirectories…

find . -name "*.txt" -type f -exec sed -i 's/www.trendics.com/tools.trendics.com/' {} \;

Here is how this works…

• The dot after the find command specifies to start in the current directory
• The -maxdepth 1 specifies to only include the current directory
• The -name "*.txt" switch specifies to only find txt files
• The -type f specifies to only match files
• The -exec xyz {} \; specifies to execute xyz for each file where xyz is a sed command specifying to substitute “tools.trendics.com” for “www.trendics.com”

Refer to the documentation on the find and sed for additional options.

Before you run this on your important files, I’d recommend backing up your important files and I’d recommend testing your modified find command in a subdirectory with test files to ensure your find command is working as expected.

First, let’s just store off the netstat output so we don’t have to retrieve it repeatedly…

echo "Saving current connections..."
netstat -nta > /tmp/netstat.txt

Next, let’s see the number of connections per IP address by extracting the IP address from the netstat output, counting the number of times each IP address was listed in the netstat output, and printing the top 10 (adjust head -10 appropriately if you’d like to see more than the top 10)…

echo "Number of connections per IP..."
cut -b 49-75 /tmp/netstat.txt | grep -o -P "\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b" | sort | uniq -c | sort -n -r -k 1,7 | head -10

If there is an excessive number of connections from a single IP, you may want to take action to block this IP.

Next, let’s see the number of connections in different states by extracting the state from the netstat output, and counting the number of times each state was listed in the netstat output…

echo "States of connections..."
cut -b 77-90 /tmp/netstat.txt | sort | uniq -c

The key reason to run this is to understand if you might be under a syn-flood attack. If you see an excessive number of connections in the SYN_RECV state, you may be under a syn-flood attack. It is normally unusual to see a high percentage of connections in a SYN_RECV state.

Next, let’s see the IP addresses generating connections currently in the SYN_RECV state by grepping the netstat output appropriately, counting the number of times each IP address was listed in the netstat output, and printing the top 10 (adjust head -10 appropriately if you’d like to see more than the top 10)…

echo "Number of SYN_RECV connections per IP..."
fgrep "SYN_RECV" /tmp/netstat.txt | cut -b 49-75 | cut -d ':' -f1 | sort | uniq -c | sort -n -r -k 1,7 | head -10

If the above command does not return any output, you simply do not have any connections in the SYN_RECV state which is good.

With all the above commands, we’ve analyzed the current connections to the server; however, this is not enough as you may not see IP addresses establishing lots of short connections. To analyze the number of new connections established to your server, you can execute this…

echo "Count number of new connection requests over the next 100 packets..."
time tcpdump -ns 200 -c 100 '(dst port http or dst port https) and tcp[13] & 2!=0' | grep -o -P '\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}.\d{1,5}\s\>' | cut -d '.' -f 1-4 | sort | uniq -c | sort -n -r -k 1,7 | head -25

For this command, the -c parameter specified how many packets to analyze. Generally, I like to have this command run for about 30 seconds and I adjust the -c parameter to make it run over 30 seconds (very busy servers will need a much larger -c value and very light servers will need a much smaller -c value).

If the above command doesn’t ever return data, you have the -c parameter too high — basically the command blocks until the number of specified packets is received and your server has not yet received that number of packets.

In addition, the command above is only analyzing http/https connections; however, you can change the dst port http or dst port https to check other types of connections (consult the tcpdump documentation).

Also, it is useful to run all the commands above under normal conditions so you have a baseline to compare against when something abnormal occurs.

# Don't forget to execute this in the right directory!
cd /the-right-directory

# Let's preview what will get deleted just to be sure
find . -maxdepth 1 -name "*" -type f -exec echo {} \;

# OK, let's do this
find . -maxdepth 1 -name "*" -type f -exec rm -v {} \;

What is this last find doing? The dot after find says to do this in the current directory, the -maxdepth 1 parameter says not to recurse into subdirectories, the -name "*" parameter says to match any file, the -type f parameter says to do this only on files, and the -exec xyz \; says to execute xyz for each file where xyz is a rm -v command with {} being the name of the current file.

Important: This is doing a permanent wildcard delete in a directory. Double check you understand what you are doing before you execute this command.

1. 50 largest files…

   find / -path '/proc' -prune -o -size +1000k -printf '%s %p\n' | sort -k1 -g -r | head -50

2. 50 largest directories (w/o files in subdirectories)…

   du -kS / | sort -k1 -g -r | fgrep -v '/proc' | head -50

3. 50 largest directories (with files in subdirectories)…

   du -k / | sort -k1 -g -r | fgrep -v '/proc' | head -50

This post is not intended to be a tutorial on the Linux shell and how to use each of the referenced commands; instead, these are snippets I’ve found useful over the years to analyze disk space.

Here is some of the output from running the first command above on one of our servers…

236272746 /root/sb_opt_2007_09_16.tar.gz
123542817 /opt/sb/sb-ext.zip
116717016 /home/twall/sb-ext.tar.gz
116717016 /home/kjohn/sb-ext.tar.gz
74840792 /home/kjohn/ib-ext.tar.gz
74840708 /home/twall/ib-ext.tar.gz
64638719 /root/jdk-6u3-linux-i586-rpm.bin
56835774 /root/jdk-6u3-linux-i586.rpm
54337152 /usr/lib/locale/locale-archive
48276960 /usr/java/jdk1.6.0_03/jre/lib/rt.jar
48117502 /root/install/jdk-1_5_0_12-linux-i586.rpm
39996663 /usr/java/jdk1.5.0_12/jre/lib/rt.jar
29923467 /root/sb_ext.tar.gz
24114161 /root/sb_cvsroot_2007_09_18.tar.gz
21577776 /usr/lib/libgcj.so.7.0.0
19673088 /var/lib/rpm/Packages
19077102 /root/install/MySQL-server-5.0.45-0.i386.rpm
18874368 /var/lib/mysql/ibdata1
18671152 /usr/java/jdk1.6.0_03/src.zip

Note that these commands can take a long time to run and generate a lot of load — be careful running these on production systems.