Professional Web Applications Themes

No Subject - Linux Setup, Configuration & Administration

Subject: RAID5 performance Lines: 30 Date: Thu, 11 Dec 2003 09:45:02 GMT NNTP-Posting-Host: 69.34.91.102 X-Complaints-To: net X-Trace: newsread1.news.pas.earthlink.net 1071135902 69.34.91.102 (Thu, 11 Dec 2003 01:45:02 PST) NNTP-Posting-Date: Thu, 11 Dec 2003 01:45:02 PST Xref: intern1.nntp.aus1.giganews.com comp.os.linux.setup:450934 I'm setting up a database server with a LSI Logic U320 RAID controller, using the megaraid2 driver. So far, I've only tested simple read and write performance. I'm not particularly impressed using the defaults. There are six Ultrastar 146Z10 drives, each capable of up to 66MB/sec sustained throughput (near track 0 - goes down to around 40MB/sec towards the last track). With the six ...

  1. #1

    Default No Subject

    Subject: RAID5 performance
    Lines: 30
    Date: Thu, 11 Dec 2003 09:45:02 GMT
    NNTP-Posting-Host: 69.34.91.102
    X-Complaints-To: net
    X-Trace: newsread1.news.pas.earthlink.net 1071135902 69.34.91.102 (Thu, 11 Dec 2003 01:45:02 PST)
    NNTP-Posting-Date: Thu, 11 Dec 2003 01:45:02 PST
    Xref: intern1.nntp.aus1.giganews.com comp.os.linux.setup:450934

    I'm setting up a database server with a LSI Logic U320 RAID controller,
    using the megaraid2 driver.

    So far, I've only tested simple read and write performance. I'm not
    particularly impressed using the defaults.

    There are six Ultrastar 146Z10 drives, each capable of up to 66MB/sec
    sustained throughput (near track 0 - goes down to around 40MB/sec towards
    the last track).

    With the six drives setup as RAID5 with the default 64KB stripe size, the
    write performance seems to around 8 to 12MB/sec. That's horribly slow. I
    know writing parity will slow down writes with RAID5, but it shouldn't
    slow them down to a fraction of the performance of a single drive.

    Read performance is better, about 150MB/sec. However, that's an average
    of only 25MB/sec per drive, which is less than half the transfer rate for
    the region of the drives that the read testing was done from.

    I'm hoping someone knows off hand how best to tune this, so I don't have
    to spend a whole lot of time testing different stripe sizes and cache
    settings.


    --
    - Mike

    Remove 'spambegone.net' and reverse to send e-mail.


    Mike Guest

  2. #2

    Default No Subject

    Subject: 64-bit Opteron kernel with Slackware 9.0?
    Lines: 20
    Date: Fri, 12 Dec 2003 12:17:33 GMT
    NNTP-Posting-Host: 69.34.89.106
    X-Complaints-To: net
    X-Trace: newsread1.news.pas.earthlink.net 1071231453 69.34.89.106 (Fri, 12 Dec 2003 04:17:33 PST)
    NNTP-Posting-Date: Fri, 12 Dec 2003 04:17:33 PST
    Xref: intern1.nntp.aus1.giganews.com comp.os.linux.setup:450975

    Anyone know what steps need to be followed to get a Slackware 9.0 Opteron
    system booting from a 64-bit kernel and with a native 64-bit build of gcc
    available?

    I've downloaded the cross-compiler stuff from suse.com, and was able to
    build a x86_64 kernel that booted, but no modules worked, and I was not
    able to build gcc 3.3.2 as a native x86_64 compiler (with which I hoped to
    generate a more fully functional kernel).

    The system (dual-Opteron with 4GB of RAM) is running fine, but I'd really
    like to be able to run the 64-bit version of MySQL with a 64-bit kernel.
    Most of the rest can remain 32-bit.


    --
    - Mike

    Remove 'spambegone.net' and reverse to send e-mail.


    Mike Guest

  3. #3

    Default No Subject

    Subject: Regarding RAID5 with MegaRAID
    Lines: 16
    Date: Fri, 12 Dec 2003 22:49:34 GMT
    NNTP-Posting-Host: 69.34.89.106
    X-Complaints-To: net
    X-Trace: newsread1.news.pas.earthlink.net 1071269374 69.34.89.106 (Fri, 12 Dec 2003 14:49:34 PST)
    NNTP-Posting-Date: Fri, 12 Dec 2003 14:49:34 PST
    Xref: intern1.nntp.aus1.giganews.com comp.os.linux.setup:450997

    In case anyone is curious, I found that using five drives instead of six
    (making the sixth a hot spare) produces greatly improved performance.

    With write back caching and cached I/O enabled, I get a write speed of
    about 75MB/sec (as measured by how long it took to copy a 1.7GB file that
    was in the read buffer of another drive already), and a read speed of over
    150MB/sec (as measured by unmounting and remounting to the file system to
    clear buffers, then reading it with a quick program made for the task).


    --
    - Mike

    Remove 'spambegone.net' and reverse to send e-mail.


    Mike Guest

  4. #4

    Default No Subject

    Subject: Re: RAID5 performance
    Lines: 47
    Date: Fri, 12 Dec 2003 23:09:05 GMT
    NNTP-Posting-Host: 69.34.89.106
    X-Complaints-To: net
    X-Trace: newsread1.news.pas.earthlink.net 1071270545 69.34.89.106 (Fri, 12 Dec 2003 15:09:05 PST)
    NNTP-Posting-Date: Fri, 12 Dec 2003 15:09:05 PST
    Xref: intern1.nntp.aus1.giganews.com comp.os.linux.setup:450998

    On Fri, 12 Dec 2003 20:10:14 GMT, P.T. Breuer wrote:
     [/ref]
     [/ref]
    >
    >You'd expect it to be equal to the bandwidth available on your buses and
    >controllers. How are these arranged? Are these scsi or IDE? If IDE, then
    >you're talking three buses, no? And you can only access one at a time
    >on each bus. So I'd expect the bandwidth to be three times 60MB/s or
    >so, which is what you are getting.[/ref]

    "IDE" and "RAID" do not belong together in the same sentence.

    The drives are U320 SCSI, one a single channel. At full transfer speed
    they would saturate the channel, but I don't expect that to happen often.


    Being SCSI, all drives can be read simultaneously.
     [/ref]
    >
    >Eh?[/ref]

    Hard drives do not have the same media transfer rate from one track to the
    next. Track 0 is much faster than the last track, typically by about
    100%. The data being read was at the beginning of the volume, meaning it
    should have been stored physically near track 0 on each drive.
     

    The (very) simple test involved copying a 1.7GB file to the volume, after
    it had been read completely and still resided completely in the read
    buffer of the source drive (the box has 4GB of RAM), leaving the read
    speed of that drive out of the equation.

    Not a scientific test, to be sure, but rough and ready.


    --
    - Mike

    Remove 'spambegone.net' and reverse to send e-mail.


    Mike Guest

  5. #5

    Default Re: RAID5 performance

    Mike Ruskai <net> wrote: 
    [..]
     

    Can't really (for sure) remember if it was with such an megaraid.o
    powered controller, however write performance was even lower. The
    only way to get some throughput was running RAID 10 (1+0) on the
    controller. I'd go for RAID1 on the system disk anyway.

    If this works better, I'd drop the author a mail about the
    problem.

    --
    Michael Heiming

    Remove +SIGNS and www. if you expect an answer, sorry for
    inconvenience, but I get tons of SPAM
    Michael Guest

  6. #6

    Default Re: RAID5 performance

    "Mike Ruskai" <net> said: 

    How are you measuring? This might well depend on the write size you're
    using.

    So, with small writes (below the stripe size - or was it below
    (ndisks-1)*stripesz), the disk subsystem needs to do read-modify-
    calculate-write cycles to update the parity. When you write big
    enough chunks at a time, the read-modify -part becomes obsolete as
    what is being written overwrites all the old data within the stripe,
    so as there's no need to read the old data from the stripe to calculate
    the new parity, the RAID controller can just calculate the new parity
    and write the full stripe (incl. new parity).
    --
    Wolf a.k.a. Juha Laiho Espoo, Finland
    (GC 3.0) GIT d- s+: a C++ ULSH++++$ P++ L+++ E- W+$ N++ !K w !O !M V
    PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
    "...cancel my subscription to the resurrection!" (Jim Morrison)
    Juha Guest

  7. #7

    Default Re: RAID5 performance

    Juha Laiho <fi> wrote: [/ref]

    Eh? Won't it have to read the parity disk, read the target disk, then
    write the target disk, then write the parity disk (yesss, I know it's
    not a disk but a stripe)?

    That would make it four times as slow on write to start with. And
    that'sif they do it the way I just suggested. They could read all the
    data disks instead of reading the parity disk :-).
     [/ref]

    You'd expect it to be equal to the bandwidth available on your buses and
    controllers. How are these arranged? Are these scsi or IDE? If IDE, then
    you're talking three buses, no? And you can only access one at a time
    on each bus. So I'd expect the bandwidth to be three times 60MB/s or
    so, which is what you are getting.
     [/ref]

    Eh?
     

    Yes.
     

    I don't get that! Surely he's not rewriting the same data area again
    and again!
     

    Eh? He'd have to read the data off all disks to find the current parity
    without reading the parity stripe itself. I don't get the argument Are
    you saying he can build up a picture in cache of the parity as he
    writes? That's not clear to me.

    Peter
    P.T. Guest

  8. #8

    Default Re: RAID5 performance

    Mike Ruskai <net> wrote: 
    >
    > The (very) simple test involved copying a 1.7GB file to the volume, after
    > it had been read completely and still resided completely in the read
    > buffer of the source drive (the box has 4GB of RAM), leaving the read[/ref]

    Oh, I see. I imagined the whole file could not be cached in ram (I've
    never had my hands on as much ram as that!) given the figure of 17GB,
    hence my puzzlement.
     

    Peter
    P.T. Guest

  9. #9

    Default Re: RAID5 performance

    On Thu, 11 Dec 2003 09:45:02 +0000, Mike Ruskai wrote:
     

    This may be too late for you, but here are my recommendations:

    1. Do not use RAID 5, it is slow on write performance. Use RAID 10 if
    your controller supports it (it probably does). At work our software is
    basically a database application and on a RAID 5 it crawls but RAID 10 it
    is much, much better.

    2. Use a smaller stripe size. 16KB would be good. This is what we have
    found to work best. If we are setting up the server for our clients on
    UNIX/Linux we go 8 KB or 16KB for the stripe.

    Don


    -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
    http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
    -----== Over 100,000 Newsgroups - 19 Different Servers! =-----
    D Guest

  10. #10

    Default Re: RAID5 performance

    it.uc3m.es (P.T. Breuer) said: 
    >
    >Yes.

    >
    >I don't get that! Surely he's not rewriting the same data area again
    >and again![/ref]

    No, not same. But all data on RAID5 is arranged on stripes. So, there's
    a set of data stripes corresponding to a single parity stripe. Whenever
    your write is so large that it completely overwrites a complete set of
    data stripes corresponding to a single parity stripe, then the parity
    can be calculated purely based on the written data - so the old parity
    (and rest of the old stripe) does not need to be read from the disks.
    --
    Wolf a.k.a. Juha Laiho Espoo, Finland
    (GC 3.0) GIT d- s+: a C++ ULSH++++$ P++ L+++ E- W+$ N++ !K w !O !M V
    PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
    "...cancel my subscription to the resurrection!" (Jim Morrison)
    Juha Guest

Similar Threads

  1. (no subject)
    By Nelson A. de Oliveira in forum Macromedia Fireworks
    Replies: 19
    Last Post: December 2nd, 11:48 PM
  2. No Subject
    By in forum Macromedia ColdFusion
    Replies: 0
    Last Post: January 1st, 12:00 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139