Showing posts with label Storage. Show all posts
Showing posts with label Storage. Show all posts

Tuesday, 10 June 2014

Partitioning and Mounting Disks For Security

This is part of a series of articles on Red Hat Server Hardening.

By creating different partitions, data can be separated and grouped. When an unexpected accident occurs, only data on that partition will be damaged, while the data on other partitions will survive.

During the initial installation, mount filesystems with user-writeable directories, such as the following, on separate partitions:
  • /usr
  • /home
  • /var
  • /var/tmp
  • /tmp

Apache and FTP server root directories should also be mounted on separate partitions.

To limit user access to filesystems, add the mount options from the following table to the filesystems configuration in /etc/fstab . The defaults option is equal to rw,suid,dev,exec,auto,nouser,async.

OptionDescription
noexecPrevents the execution of binaries (although scripts will not be prevented from running).
nosuidPrevents the setuid bit from having an effect.
nodevPrevents the use of device files.

Modify the /boot directory to be read only (ro). This reduces the risk of unauthorized modification of critical boot files.

For example, to modify the /etc/fstab entry to limit user access on /dev/sda5 (ftp server root directory):

Find the line that reads:
/dev/sda5  /ftpdata          ext3    defaults 1 2
And change it to:
/dev/sda5  /ftpdata          ext3    defaults,nosuid,nodev,noexec 1 2

Saturday, 31 May 2014

Display the Amount of Free Space on Each Filesystem

Another old script that I came across. This one displays the amount of free space on each filesystem on a server, and displays a warning if it falls below a certain level. The output can be displayed in html format for sending as an email (via the htmmail script).

The script was written for AIX - one day I might adapt it for Linux, which has the very useful free command, but without the bells and whistle of the script below.

Save the file as fsfree.ksh, and run it from cron on a daily basis for reports on filesystem usage.

#!/bin/ksh
##################################################################
#
# fsfree.ksh
#
# Purpose: To display free space on each filesystem on a server
#          and to warn if it falls below a certain percentage.
#
# Return status:
#            0  All filesystem space is above the limit
#            1  One or more filesystems are below the limit
#            2  The script has not been correctly executed
#
# Syntax:  fsfree.ksh [ -a ] [ -g | -m | -p ] [ -h ] [-q ]
#                   [ -l limit | -f file ]
#
#          Flags
#           -a  Displays all filesystems
#
#               By default, the script will display only those
#               filesystems which are below the limit.
#
#           -f  Use the specified file to define the limits
#               for the individual filesystems
#
#               The format of the file should be:
#                    Filesystem   Limit
#
#               An initial file may be generated using the
#               free command as follows:
#
#                    fsfree.ksh -pal 0 > limitfile.dat
#
#               The file should then be edited as appropriate
#     
#           -g  Displays total and free space in Gigabytes
#
#               Space is displayed in Kilobytes by default
#
#           -h  Output is in html format
#
#           -l  When the percentage free on the filesystem falls
#               below the value of 'limit', the filesystem will be
#               highlighted on the output.
#
#               The default limit is 10%
#
#           -m  Displays total and free space in Megabytes
#
#               Space is displayed in Kilobytes by default
#
#           -p  Total and free space is not displayed
#
#           -q  Quiet mode. No output is displayed.
#
# Author:  Douglas Milne
# Date:    13th May 2008
#
##################################################################
#
# Version 1.0    Initial Release
#
##################################################################

##################################################################
#
# VARIABLES Section
#
#
# Set default variables
#
dispall=false
htmflag=false
pcntonly=false
quietmode=false
limitsource=default
divisor=1
stype="Kb"
scale=0
dlimit=10
returnstat=0
errstring=""

# Parse the flags and change default variables accordingly
while getopts :ahpgml:f:q value
do
   case $value in
      a) dispall=true
         ;;
      h) htmflag=true
         ;;
      p) pcntonly=true
         ;;
      g) divisor=1048576
         stype="Gb"
         scale=3
         ;;
      m) divisor=1024
         stype="Mb"
         scale=3
         ;;
      l) if $(echo $OPTARG | grep -q [0-9])
         then
            dlimit=$OPTARG
            limitsource=limit
         else
            errstring="Limit must be a number between 1 and 100"
            dlimit=""
         fi
         ;;
      f) limitfile=$OPTARG
         limitsource=file
         if [ ! -f $limitfile ]
         then
            errstring="The file $limitfile does not exist"
         fi
         ;;
      q) quietmode=true
         ;;
     \?) errstring="$0: unknown option $OPTARG"
         ;;
   esac
   if [ "$value" == ":" ]
   then
      case $OPTARG in
         l) errstring="Limit argument must be included and must be a number between 1 and 100"
            ;;
         f) errstring="The name of the file may not be blank"
            ;;
     esac
   fi
done
    
shift $(expr $OPTIND - 1)

##################################################################
#
# FUNCTIONS Section
#

syntaxerr ()
##################################################################
#
#  Syntax:      syntaxerr error
#
#  error        Text describing the error
#
#  Writes a copy of the error and the syntax of the script
#  to stderr.
#
##################################################################
{
   err=$1
   echo "$err\n\nSyntax:  fsfree.ksh [ -a ] [ -g | -m | -p ] [ -h ] [-q ]\n                    [ -l limit | -f file ]\n" >&2
   exit 2
}

limitdef ()
##################################################################
#
#  Syntax:      limitdef filesystem
#
#  filesystem   A filesystem name
#
#  The function will return the limit required for the given
#  filesystem. This may be the default limit, the limit defined
#  from the command line, or the limit defined in the limitfile
#
##################################################################
{
   fs=$1
   case $limitsource in
      default) echo $dlimit
               ;;
        limit) echo $dlimit
               ;;
         file) lim=$(grep ^/ $limitfile | grep "^$fs " | sort | head -1 | awk '{print $2}' | sed -e's/\%//g')
               if [ "$lim" == "" ]
               then
                  echo $dlimit
               else
                  echo $lim
               fi
               ;;
   esac
}

display ()
##################################################################
#
#  Syntax:      display text
#
#  text         The text to display
#
#  The function will print the text to the standard output, but
#  only if the script is not running in quiet mode
#
##################################################################
{
   text=$1
   if [ $quietmode == false ]
   then
      echo "$text"
   fi
}

##################################################################
#
# MAIN Section
#

# Error checking

if [ "$errstring" != "" ]
then
   syntaxerr "$errstring"
fi

# Display the header
if [ $htmflag == true ]
then

# If html format is required, then create the head and body of the
# html page.
#
# A table is used to format the output. Define the table, and output
# the header.
#
   display "<html><head>"

# Instead of below, it might be useful to cat in a standard css file instead
   display "<style> th{font:10pt Arial;font-weight:bold;color:blue;background-color:LightBlue;padding-left:10px;padding-right:10px;} p{font:10pt Arial;} li{font:10ptArial;} td{font:10pt Arial;padding-left:10px;padding-right:10px;} .normal{color:black;} .alert{color:red;font-weight:bold;} .code{font:10pt Courier;color:black;}</style>"


   display "</head><body>"
   display "<p class=\"normal\">$(hostname) filesystem status at $(date)"
   if [ $limitsource != "file" ]
   then
      display "<br><br>Limit is $dlimit%"
   fi
   display "</p>"
else
   display "\n$(hostname) filesystem status at $(date)\n"
   if [ $limitsource != "file" ]
   then
      display "Limit is $dlimit%\n"
   fi
fi

if [ $htmflag == true ]
then

#
# A table is used to format the output. Define the table, and output
# the header.
#
   display "<table>"
   display "<tr>"
   display "<th style=\"text-align: left\">Mount Point</th>"
   if [ $pcntonly = false ]
   then
#
# Only output the total Free and Total Size headers if the -p flag
# has not been used
#
      display "<th style=\"text-align: right\">Total<br> Free ($stype)</th>"
      display "<th style=\"text-align: right\">Total<br> Size ($stype)</th>"
   fi
   display "<th style=\"text-align: right\">%age<br> Free</th>"
   if [ $limitsource == "file" ]
   then
      display "<th style=\"text-align: right\">%age<br>Limit</th>"
   fi
   display "</tr>"
else


# If html format is not required, then output the header as standard
   boldon=$(tput smso)
   boldoff=$(tput rmso)
   typeset -L20 mntpnt
   typeset -R11 free total
   typeset -R3 pcnt limit
   display "Mounted on          \c"
   if [ $pcntonly = false ]
   then
#
# Only output the total Free and Total Size headers if the -p flag
# has not been used
#
      display "    Free ($stype)   Total ($stype)\c"
   fi
   display " %Free\c"
   if [ $limitsource == "file" ]
   then
      display " Limit"
   else
      display
   fi
fi

df -k > /tmp/freedf

#
# For each filesystem in turn, read the mountpoint, total free
# space and total size
#
cat /tmp/freedf | tail +2 | awk '{print $7,$3,$2}' | while
read mntpnt free total
do
# If numeric values have been read, then calculate the output
   if $( echo $free | grep -q [0-9])
   then

# 1) Calculate the percentage of free space
      let pcnt=free\*100/total
# 2) Calculate the free and total space as Kb, Mb or Gb as required
      free=$(echo "scale=$scale; $free / $divisor" | bc)
      total=$(echo "scale=$scale; $total / $divisor" | bc)

      limit=$(limitdef $mntpnt)
# If the percentage free is less than the limit allowed, then switch on
# highlighting. For html output, use the alert class. For standard output,
# use bold type. Set the return status to 1.
      if (( pcnt <= limit ))
      then
         if [ $htmflag == true ]
         then
            class=alert
         else
            display "$boldon\c"
         fi
         returnstat=1
      else

# If the percentage free is greater than the limit allowed, then use the
# normal html class. This irrelevalnt for non-html output
         class=normal
      fi
      if [ $dispall == true -o $pcnt -le $limit ]
      then
         if [ $htmflag == true ]
         then

# If using html output, then create a table row
            display "<tr>"
            display "<!-- Mount Point -->"
            display "<td class=\"$class\">$mntpnt</td>"
            if [ $pcntonly = false ]
            then

# Only display free and total size if they are required
 
               display "<!-- Free Space -->"
               display "<td class=\"$class\" style=\"text-align: right;\">$free</td>"
               display "<!-- Total Space -->"
               display "<td class=\"$class\" style=\"text-align: right;\">$total</td>"
            fi
            display "<!-- Percentage Free -->"
            display "<td class=\"$class\" style=\"text-align: right;\">$pcnt%</td>"
            if [ $limitsource == "file" ]
            then
               display "<!-- Limit -->"
               display "<td class=\"$class\" style=\"text-align: right\">$limit%</td>"
            fi
            display "</tr>"
         else
# If using standard output, write a row.
            display "$mntpnt  \c"
            if [ $pcntonly = false ]
# Only display free and total size if they are required
            then
               display "$free  $total  \c"
            fi
            display "$pcnt%  \c"
            if [ $limitsource == "file" ]
            then
               display "$limit%\c"
            fi

            if (( pcnt <= limit ))
            then
               display "$boldoff"
            else
               display
            fi
         fi
      fi
   else
# if the values read were not numeric, then do not perform any calcualtions
# simply output them as read.
      if [ $dispall == true ]
      then
         if [ $htmflag == true ]
         then
            class="normal"
            display "<tr>"
            display "<!-- Non numeric values read -->"
            display "<td class=\"$class\">$mntpnt</td>"
            if [ $pcntonly = false ]
            then
               display "<td class=\"$class\" style=\"text-align: right;\">$free</td>"
               display "<td class=\"$class\" style=\"text-align: right;\">$total</td>"
            fi
            display "</tr>"
         else
            if [ $pcntonly = false ]
            then
               display "$mntpnt  $free  $total"
            else
               display "$mntpnt"
            fi
         fi
      fi
   fi
done

if [ $htmflag == true ]
then
# If using html output, close the table and the html page
   display "</table></body></html>"
fi

return $returnstat

#
#############################################################

Monday, 26 May 2014

What is RAID?

RAID stands for Redundant Array of Inexpensive (or Independent) Disks. In simple terms, RAID is a method of storing data across several disks. There are several reasons why this is a good idea:

  • Provides better performance
  • Provides redundancy in the storage system, thus providing greater reliability
  • Allows several smaller physical hard drive to be treated as one large logical hard drive
  • Allows greater flexibility in adding or removing hard drives

RAID Data Distribution Techniques

RAID is achieved via a combination of a variety of methods of distributing data across a number of disks. Individually, these allow for faster access, or for data recovery. Different RAID levels use different combinations depending on the requirements of the system requirements.

Striping

Striping distributes the data across 2 or more disks. This allows for faster access to the data as the two disks can be reading or writing data simultaneously

Mirroring

Mirroring puts a copy of the data on a second disk. New disks must be added to the system in pairs - one for data and one for the mirror. If a disk fails, the data can be recovered from the mirror.

Parity

Parity performs an Exclusive-OR (XOR) between bytes on two disks and writes the results to a third. If any of the three disks fails, the missing data can be reconstructed by performing an XOR between the two remaining disks.

RAID Levels

RAID may be implemented at different levels, depending on the levels of redundancy and performance requirements of the overriding system. These two factors trade against each other: A higher level of redundancy reduces overall performance, but increases the safety of the data and the reliability of the system.

Linear

  • Treats RAID hard drives as one virtual drive
  • No striping, mirroring or parity reconstruction
  • Storage is sequential
  • No recovery capability

0 - Striping

  • Implements disk striping across drives with no redundancy (no mirroring and no parity)
  • Very efficient
  • Standardized stripes across drives
  • Faster access
  • Requires a minimum of 2 disks
  • Should not be used for any critical system

1 - Mirroring

  • Implements redundancy through mirroring  (but no striping. and no parity)
  • The same data is written to each RAID drive
  • Each disk has a complete copy of all of the data
  • If one or more of the disks fail, then the others still have the data
  • Very safe, but inefficient
  • Requires a minimum of 2 disks
  • Good performance

5 - Distributed Parity

  • Implements data reconstruction capability using parity information. Blocks are striped and parity information is therefore distributed across all drives
  • An alternative to mirroring
  • Parity information is saved instead of full data duplication
  • Requires a minimum of 3 disks
  • Good redundancy
  • Provides a good balance between performance and redundancy, and can be very cost-effective.
  • Write operations will be slow.
  • Use for systems that are heavily read oriented.

10 (or 1+0) - Striped Mirroring

  • Implements redundancy through mirroring  (but no parity)
  • A striped copy of the data is written to half of the RAID drives, then mirrored to the other half
  • Each half has a complete striped copy of all of the data
  • If one or more of the disks fail, then the others still have the data
  • Requires a minimum of 4 disks
  • Good performance and good redundancy
  • Easily the best option for mission critical applications