Dell PERC RAID controller and RAID 10

--- SEARCH TAGGING ---
TAG000000000000000 - SEANRYAN
TAGA00000000000001 - Public
TAGB00000000000001 - Main Area
TAG300100000000000 - Hardware, I/O sub-system
--- SEARCH TAGGING ---

Subject:  Dell PERC RAID controller and RAID 10
Hardware:  Dell Power Edge Server line (I.E. PE 1650, PE2550, PE8400...)
Creation Date:  03-18-04
Last Revision Date:
References:  "Dell OpenManage Array Manager 2.7 User's Guide", Adaptec On-line help, and various forum 
postings by various authors.

     This is in reference to the SCSI RAID controllers that Dell ships in their Power Edge server Line.  
In a nutshell, Although Dell claims that their PERC RAID controller line supports a variety of RAID 
levels including RAID 10 this is not actually the case.  They implement what is known as 
"RAID 1 + concatenation" which does not yield the performance benefits of striping.

     On a Dell PE6400 with a PERC3/QC driving a PV201S I configured what according to Dell was a RAID 10 
array.  I quickly noticed that I/O throughput on this logical volume was dramatically slower that 
anticipated.  After a great deal system testing and monitoring I could find no cause for the problem.  
While most of Dell’s documentation refers to RAID 10 further research produced "Dell OpenManage Array 
Manager 2.7 User's Guide" dated January 2001 which discusses their use of "RAID 1 + concatenation".  
Here is an excerpt:

"Concatenation 
Concatenates n disks as one large virtual disk with a capacity of n disks.  Data fills up the first disk 
before it is written to the second disk  (So there goes all the benefits of multi-spindle performance,no 
wonder that folks are not seeing the performance referred to in PeterWong’s 220S whitepaper).  No 
redundancy data is kept. When a disk fails, the large virtual disk fails.  No performance gain.  No 
redundancy
Note: On the PERC 2/SC, 2/DC, 3/DCL, 3/DC, and 3/QC controllers, RAID-10 is implemented as 
RAID-1 Concatenated. See the next topic RAID Level 1-Concatenated (Concatenated mirror). RAID Level 
1-Concatenated (Concatenated mirror)
RAID-10 on PERC 2/SC, 2/DC, 3/DCL, 3/DC, and 3/QC controllers is implemented as RAID-1 Concatenated.
RAID-1 Concatenated is a RAID-1 array that spans across more than a single pair of array disks. This 
combines the advantages of concatenation with the redundancy of RAID-1. No striping is involved in this 
RAID type. 
Also, RAID-1 Concatenated can be implemented on hardware that supports only RAID-1 by creating multiple 
RAID-1 virtual disks, upgrading the virtual disks to dynamic disks, and then using spanning to 
concatenate all of the RAID-1 virtual disks into one large dynamic volume."

     It seems that the PERC3 controllers are actually Adaptec 39160 SCSI controllers to which Dell has 
added a "daughter processor" or "daughter card", depending on where you read.  According to Adaptec’s 
on-line help the 39160 is a SCSI controller but not a RAID controller.  It is not capable of any level 
of RAID.  Quote from the Adaptec online help:

"Q:  Can the Adaptec SCSI Card 39160 do hardware RAID?

A:  The card is not capable of hardware level RAID.  If you are looking for Ultra160 Dual Channel SCSI 
    card that is capable of hardware RAID, you might want to look into Ultra160 or Ultra320 SCSI RAID 
    from Adaptec."

  This leaves the "daughter processor" added by Dell to handle the RAID implementation.  Dell’s own 
contradictory definitions of RAID 10, RAID 0+1, and RAID 1 + concatenation only seems to add mud to 
the already murky water.  

     To illustrate lets compare RAID 0+1 to RAID 1 + concatenation. 
 
     RAID 0+1 stripes your data across multiple spindles and then mirrors that stripe onto another set 
of disks.  For example if you have 8 disks you would stripe across the first 4 disks and then mirror the 
first 4 onto the second 4.  Now add in that you have a stripe size, or stripe width, of 64 kb.  When you 
write data to this volume it writes 64 kb to disk 1 (and mirrors that data on disk 5), then writes the 
next 64 kb to disk 2 (and mirrors that data on disk 6), then 64 kb to disk 3 (mirror on disk 7), then 
64 kb to disk 4 (mirror on disk 8), then 64 kb to disk 1 (mirror on disk 5) and so on.  This very 
effectively distributes sequential disk reads and writes across multiple physical disks in small, 64 kb, 
units and (depending on the controller and some other configuration concerns) performs these reads and 
writes in parallel rather than concentrating all activity on 1 physical disk.  Not only are sequential 
I/Os spread across the width of the strip but the odds are very good that non-sequential I/Os will be 
well distributed as well.  The end result is faster reads and writes.

     RAID 1 + concatenation gives you mirroring, hence the RAID 1 part of the name, but replaces striping 
across multiple disks with concatenation of disks.  Given the same example of 8 disks you would get 
something like this.  When you write data to this volume it starts writing on disk 1 (and mirrors that 
data on disk 5) just like with RAID 0+1 but instead of only writing the first 64 kb to disk 1 and the 
next 64 kb to disk 2 RAID 1 + concatenation will continue writing to disk 1 until it is full.  Then and 
only then will it begin writing to disk 2.  While this allows you to create logical volumes that are 
larger that you physical disk size it does absolutely NOTHING to distribute the I/O load across multiple 
disks.  This is the reason you do not get the performance gains that probably drove you to chose a RAID 
implementation that combined stripping for performance with mirroring for redundancy in the first place.

     Bottom line:  Dell’s PERC controller DOES NOT support RAID 10 or RAID 0+1.  With Dell’s 
implementation of RAID 1 + concatenation you WILL NOT see the performance gains of stripes which span 
multiple spindles.  This is not to say Dell or any of their products are bad or should be avoided.  In 
fact I am a huge proponent of Dell hardware for a variety of reason.  The point is to know what you are 
buying and make sure it provides the performance you require.  If you are looking for RAID 0, 1, 5, or 
1 + concatenation the PERC line of RAID controllers are very reliable and boast good performance 
statistics combined with intuitive interface options.  It just doesn’t support RAID 10 as advertised! 

I've been lucky enough to work on UNIX platforms for number of years now.  This 
can be an issue for LINUX users.

Recently I came across the following document:

   http://technet.oracl...pdf/oow2000_sane.pdf

It talks about "S.A.M.E methodology". SAME stands for Stripe and Mirror 
Everything.

Here are couple of interesting links from AskTOM in regards to this:

   http://asktom.oracle...TION_ID:396975084466
   http://asktom.oracle...TION_ID:359617936136