Fault tolerance with RAID

[Greg]

Hello. I am not that familiar with Solaris, and have to face an issue I'd
appreciate some help with.

I currently have and use Solaris 8 on a small SPARC (Ultra Enterprise 2),
with no particular problems. However, we are about to purchase a larger
machine, that will most probably have RAID support (with hot swapping). So I
have some simple questions:

1. Provided we have hardware RAID support, is there anything that has to be
done from inside the OS? I mean, is hot swapping and all the features of
RAID transparent to the sysadmin?

2. Can this whole idea of having mirrored disks be applied to the OS'
filesystems as well (eg. /usr) or is it used only for data filesystems? (eg.
/var/mail).

Sorry for not being specific enough. In general, any good resource regarding
RAID and Solaris would be mostly appreciated.

Regards,
Greg

[Sam N]

"Greg" <grp@med.uoc.gr> wrote in message news:bgam14$7m0$1@nic.grnet.gr...

> > [url]http://docs.sun.com[/url] is a good place to start, have a dig around

DiskSuite,

> > VxVm and Raid Manager.
> >

> Thanks...And one more thing: are the above part of the OS distribution or
> they are commercial software?
>
>

If you buy a RAID array from Sun I think all the licences for the above come
with it. Solstice Disk Suite (or Solaris VM as it's called in Solaris 9)
comes as part of the OS iirc - though I'm sure some other posters will know
more about this than me.

cheers

Sam N

[Anthony Mandic]

Greg wrote:

> I currently have and use Solaris 8 on a small SPARC (Ultra Enterprise 2),
> with no particular problems. However, we are about to purchase a larger
> machine, that will most probably have RAID support (with hot swapping). So I
> have some simple questions:
>
> 1. Provided we have hardware RAID support, is there anything that has to be
> done from inside the OS? I mean, is hot swapping and all the features of
> RAID transparent to the sysadmin?

You would, at the least, need to configure luns or define the
hot swap devices etc. Hardware RAID isn't very common with
Sun equipment (only a few such devices exist). More than likely
it would be software RAID that you would need to set up with SDS.
You can do that now on your U2. The software and the concepts
are the same so it won't hurt to practice. You could also use
Veritas Volume Manager but I wouldn't really bother with it.
Hardware RAID would also require use of a RAID Manager.

> 2. Can this whole idea of having mirrored disks be applied to the OS'
> filesystems as well (eg. /usr) or is it used only for data filesystems? (eg.
> /var/mail).

Filesystems are filesystems. SDS mirrors the disk partitions.
You can also have soft partitions.

> Sorry for not being specific enough. In general, any good resource regarding
> RAID and Solaris would be mostly appreciated.

Go to Sun's web site and search the docs for SDS or Solstice Disk
Suite. The version for Solaris 8 is 4.2.1.

-am © 2003

[Remco]

Hi Greg,

"Greg" <grp@med.uoc.gr> wrote in message news:bgaci2$10c$1@nic.grnet.gr...

> Hello. I am not that familiar with Solaris, and have to face an issue I'd
> appreciate some help with.
>
> I currently have and use Solaris 8 on a small SPARC (Ultra Enterprise 2),
> with no particular problems. However, we are about to purchase a larger
> machine, that will most probably have RAID support (with hot swapping). So

I

> have some simple questions:
>
> 1. Provided we have hardware RAID support, is there anything that has to

be

> done from inside the OS? I mean, is hot swapping and all the features of
> RAID transparent to the sysadmin?

Which HW RAID solution: T3, T4, A1000, SE3300, SE6200... or non sun HW
raid...
Mainly HW RAID is done on the array itself (except the A1000, you require
RM6 software)

All the latest HW RAID and JBOD arrays are having Hot-Pluggable/-swappeble
disks inside...
(JBOD= Just bunch of drives)
The RAID volumes are shown to Solaris as a normal disk with the Size of that
LUN.
If you use RAID1 or 5 and there is a disk failure, this should not be seen
by Solaris.

Mainly if you have to swap a drive you don't have to do anything within
Solaris with exception
of the A1000 ... then RM6 will help you to replace the drive

>
> 2. Can this whole idea of having mirrored disks be applied to the OS'
> filesystems as well (eg. /usr) or is it used only for data filesystems?

(eg.

> /var/mail).

The Boot disk mirroring I would put on HW raid.. but use the LVM (Disksuite)
which is
standard available for Solaris.. ( You will not have any benifit from HW
raid on a boot disk)

>
> Sorry for not being specific enough. In general, any good resource

regarding

> RAID and Solaris would be mostly appreciated.
>
> Regards,
> Greg
>
>

....

Resources.... look at [url]www.sun.com/[/url] search for storage products... and see if
it's appropriate for you...
But every product has got it's price ! ;-)

(btw I'm not working 4 Sun anymore....)

Good luck with the decision... and call Sun.. for a right sizing concept..
they would like to sell I think ;-)

Hth

[Scott Howard]

Anthony Mandic <pd@hotmail.com> wrote:

> You would, at the least, need to configure luns or define the
> hot swap devices etc. Hardware RAID isn't very common with
> Sun equipment (only a few such devices exist). More than likely

Only a few? The majority of Sun Hardware storage products are now
hardware RAID arrays. (eg, StorEdge 3510, 39x0, 6x20, 69x0, 99x0, A1000,
T3, T3+ - all of which are currently shipping products I believe)

Most of these arrays (all except the A1000) use out-of-band configuration
(either telnet, serial and/or built-in notebook) so no additional
software is required on the host.

Scott

[Anthony Mandic]

Scott Howard wrote:

> Only a few? The majority of Sun Hardware storage products are now
> hardware RAID arrays. (eg, StorEdge 3510, 39x0, 6x20, 69x0, 99x0, A1000,
> T3, T3+ - all of which are currently shipping products I believe)

OK, a half dozen or so. There's also the PCI RAID controller
card (is that still being made?).

-am © 2003

[Phil Meyer]

Greg wrote:

> 1. Provided we have hardware RAID support, is there anything that has to
> be done from inside the OS? I mean, is hot swapping and all the features
> of RAID transparent to the sysadmin?
>
> 2. Can this whole idea of having mirrored disks be applied to the OS'
> filesystems as well (eg. /usr) or is it used only for data filesystems?
> (eg. /var/mail).

In terms of making the OS itself as safe as possible from disk outages,
please reconsider the use of any software based raid. Whether Veritas
encapsulated root disks or SDS mirrored root disks you run as great a risk
of data corruption from the mirroring software as you do a catastrophic
disk failure.

Allow me to illustrate:

Go whole hog with Veritas and create a three way mirror of the root drive.
Lets imagine that one of the three drives begins to generate soft data
errors. There is a chance, that the corrupted area of the bad disk will
simply be copied to the other two drives. Lets magify the problem just
slightly by having that little bit of corruption appear across all three
drives in the mirror in a part of the drives that are currently open to
files. Make it worse by the fact that the file in question is a named pipe
to a critical process, or the inode containing /dev and you begin to see
the potential problems.

I have personally witnessed this very scenerio with both Veritas and
Disksuite. It is a very sinking feeling you get as you watch a critical
system eat itself to the point of no return.

An additional oddity with SDS is that it is very possible to replace a
failed disk, partition it correctly, and enable it, to watch SDS sync the
wrong way! Thus corrupting the good disk with whatever was on the new
disk. Not fun.

If you are going to attach to a hardware based raid device, my advice is to
set up to boot from it as well.

Your second best option is to deliver all your systems to production as FRUs
with an automated jumpstart install, and force all user/application data to
the non system disks. That way you are a simple reboot from back on-line
after a root drive failure.

It really depends upon your Service Level expectations, but software based
raid is only about 40% effective (In My Experience), and the other 60% of
the time it IS the problem. :)

[Darren Dunham]

Phil Meyer <x0pmeyer@ti.com> wrote:

> Greg wrote:

> In terms of making the OS itself as safe as possible from disk outages,
> please reconsider the use of any software based raid. Whether Veritas
> encapsulated root disks or SDS mirrored root disks you run as great a risk
> of data corruption from the mirroring software as you do a catastrophic
> disk failure.

If the disks are going to give you silent failures, why would a
"hardware mirror" be any less susceptible?

> I have personally witnessed this very scenerio with both Veritas and
> Disksuite. It is a very sinking feeling you get as you watch a critical
> system eat itself to the point of no return.

I don't see anything that makes an external array immune to such issues.

I personally think that..

1) silent corruption like this is relatively rare.
2) "non-silent" disk failures are less rare.

Given that, I would rather have *some* protection (sofware mirrors)
rather than nothing else.

> An additional oddity with SDS is that it is very possible to replace a
> failed disk, partition it correctly, and enable it, to watch SDS sync the
> wrong way! Thus corrupting the good disk with whatever was on the new
> disk. Not fun.

Ouch. I've never seen that. Is there an open bug on that?

> It really depends upon your Service Level expectations, but software based
> raid is only about 40% effective (In My Experience), and the other 60% of
> the time it IS the problem. :)

Your numbers are very different from mine.

--
Darren Dunham [email]ddunham@taos.com[/email]
Unix System Administrator Taos - The SysAdmin Company
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >

[Greg]

> The Boot disk mirroring I would put on HW raid.. but use the LVM
(Disksuite)

> which is
> standard available for Solaris.. ( You will not have any benifit from HW
> raid on a boot disk)
>

> >
> > Sorry for not being specific enough. In general, any good resource

> regarding

> > RAID and Solaris would be mostly appreciated.

(This post applies to other replies as well)

Well, it seems each one of you has a different approach :-). But thanks to
all of you.
However, in my case, data preservation is not that crucial. (I am most
easily downloading
all my data via FTP and back it up on CD frequently). The important in my
case, is the
availability of the system. In that case, I don't understand why I couldn't
benefit by
mirroring the boot disk. I personally use Red Hat's software RAID on another
machine, by
mirroring all partitions of two similar disks (including /boot - I don't
think this is allowed
by SuSE), and it seems to work fine. Maybe I did not understand completely
what you mean.

Apart from that, I believe hardware RAID would be best for my case. Not only
because it
seems to be transparent to the OS, but also because I don't have that
significant non-system
volumes to mirror (or large capacity requirements), and I am mostly
concerned on failing disks and
not silent errors as one of you describe. Using a hardware RAID-1, only a
failure on the RAID
backplane would make my system to fail (or a thunder :-)). Right? the simple
question of mine is if
the OS can be completely installed on a RAID device (SUN or non-SUN) (we
haven't yet decided
what to buy, not even considered the specifications of available SUNs).

Sorry that I follow an oversimplified approach to the problem, but it seems
to me you all deal with
far more complex situations than mine :-))

Thanks again
Greg

[Darren Dunham]

Greg <grp@med.uoc.gr> wrote:

> Apart from that, I believe hardware RAID would be best for my case. Not only
> because it
> seems to be transparent to the OS, but also because I don't have that
> significant non-system
> volumes to mirror (or large capacity requirements), and I am mostly
> concerned on failing disks and
> not silent errors as one of you describe. Using a hardware RAID-1, only a
> failure on the RAID
> backplane would make my system to fail (or a thunder :-)).

Well "hardware RAID-1" would imply either DMP or MPxIO to handle dual
paths (expensive equipment), or that the connection is through a single
controller. In that case, the failure of a controller would be a
problem, too.

With software mirroring, you can mirror across the controllers.

Right? the simple

> question of mine is if
> the OS can be completely installed on a RAID device (SUN or non-SUN) (we
> haven't yet decided
> what to buy, not even considered the specifications of available SUNs).

If you're doing hardware raid, then most of the time the work is
transparent to the OS. The storage appears as one or more LUNS, and the
OS installs.

Generally OS installations on external devices have more issues with
access (do you trust booting off an FC device if you have to upgrade the
FC drivers?) than they do with back-end configuration (RAID or not).

--
Darren Dunham [email]ddunham@taos.com[/email]
Unix System Administrator Taos - The SysAdmin Company
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >