Call for opinions on UltraSparc III perf issues

[Rich Teer]

On Wed, 30 Jul 2003, Dennis Clarke wrote:

> It is not my nature to come with bended knee and hat in hand but I
> must ask for some educated opinions here. I have a bit of a problem

Opinions are a dime a dozen on Usenet. Whether or not they're
educated is left as an exercise for the reader. :-)

> V880 processors from a floating point perspective. He hates the perf
> of the external ATABOY. None of this was my idea.

<Sarcasm>
He hates the performance of IDE drives. Surprising!
</Sarcasm>

For the rest of this post, I'm going to assume that Arnold's
application and data will not be severely crippled by the 8 GB
RAM limitation in the 280R.

> I advised him to get a 280R with the fastest UltraSparc III procs that
> money can buy. He says that he needs good floating point speed with
> software written for the UltraSparc processor. He also has let me

Your recommendation makes sense. The 1.2 GHz CPUs are substantially
faster than the 900 MHz ones he currently has.

> know that his current edition of the software is strictly a 32-bit
> binary compiled for the Sparc V8 processor. I feel that he may get a
> bit better performance if his software vendor provided a binary that
> was tweaked for the V9a but this is where I could be wrong. I have

It would certainly be worth trying. From the vendor's point of
view, it should simply be a case of tweeking Makefiles, and
rebuilding. I'm also hoping that the vendor is using Sun's
compiler, rather than gcc.

> also submitted that he should not have spent money on a 8 processor
> capable system which has only two processors in it and then expected
> blistering fast performance. I suggested that a good way to go would
> be a SunFire 280R with dual 1.2GHz processor modules. This could be

That makes sense - especially if his app doesn't multithread
much or at all. Or if he's not running many instances of his
app.

> CPU modules in the 280R. Perhaps a 64-bit architecture with 1.2GHz
> procs would be quicker than 900MHz procs in a mostly empty V880.

I agree.

> From a storage perspective I had suggested that the Sun StorEdge
> 3510 FC Array would be fairly quick when mated to the X6768A dual
> 2 GB fibre controller. If money is still an issue then the A5200 with

I've no experience here, but instinct says you're correct again.

> The last issue that I got lambasted for was my approach to patches. I
> have been living under the misguided idea that Sun releases patches
> for software because they actually fix issues with the software. I had

Nah, they do it to keep their engineers employed. :-)

> been confused or so I was told. Applying patches to the OS for things
> like CDE and X11R6 serve no purpose at all. The Recommended patch
> cluster is all that is needed and nothing else. This is where I think

WTF ARE they smoking?! So presumably, then, if they're using DiskSuite,
they don't think that the DiskSuite patch needs to be applied...

> I am truely confused. I actually apply patches for things that are
> beyond the scope of the Recommended patch cluster, even on a server
> that will probably never run graphics. This may be a misguided policy

IMHO, this is one of those situations where there are more than one
correct way to do a job (in other situations, there is only one right
way to do something, the others just being wrong).

Some people subscribe to the "install everything" school of thought,
whereas other subscribe to the "install only what you need" school.
I'm in the latter camp, which automatically means that I have much
less patches to worry about. Good thing too, as I don't have a Sun
support contract, so I can't get my hands on Contract-only patches
very easily. :-)

But even if one is in the former camp, I think appliying patches is
one of those things that experience can guide. Applying ALL of them
is one valid approach (although ISTR that when I worked at Sun, we
recommended that patches only be applied if you're hitting a problem,
the Recommended Patch cluster being an exception to this). So I guess
that what I'm saying is that it is not unreasonable to install the
patches for all the software one's installed - but I personally
wouldn't install all the software in the first place.

And I certainly think that applying security patches should be a
matter of course.

> and should be corrected. I actually apply things like the /dev/random
> feature patch ( really an RFE issue ) such that prngd is not required.
> I have even been known to hunt down patches for obscure things like
> the Open Boot Prom and the FC-AL backplane. I have been told that I
> am horribly wrong in my ways and lost. A wanderer from the faith, a
> child of superstition and a heretic.

I can't think of a single reason why one WOULDN'T apply OBP
patches and the like. I believe that prevention is better
than cure.

HTH,

--
Rich Teer, SCNA, SCSA

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: [url]http://www.rite-online.net[/url]

[Dr. David Kirkby]

Rich Teer wrote:

> I can't think of a single reason why one WOULDN'T apply OBP
> patches and the like. I believe that prevention is better
> than cure.
>
> HTH,
>
> --
> Rich Teer, SCNA, SCSA

One reason must surely be that if something goes wrong, it is remotely
possible to end up with a dead machine. The patch instructions always
warn of the risks and say the mains must not be removed. There is a
recovery procedure, but if that fails to work you need to call Sun for
help.

I'd personally not update the OBP unless the system was on a *decent*
ups. I've updated the OBP on this U80, but have chosen not to do so on
my IBM 7025 F50 server as I don't have a large enough UPS to put it
on. The mains here is not too reliable and I don't want to risk a dead
IBM. That could potentially happen if the mains failed during the
update.


--
Dr. David Kirkby,
Senior Research Fellow,
Department of Medical Physics,
University College London,
11-20 Capper St, London, WC1E 6JA.
Tel: 020 7679 6408 Fax: 020 7679 6269
Internal telephone: ext 46408
e-mail [email]davek@medphys.ucl.ac.uk[/email]

[Dr. David Kirkby]

Dennis Clarke wrote:

> I advised him to get a 280R with the fastest UltraSparc III procs that
> money can buy. He says that he needs good floating point speed with
> software written for the UltraSparc processor. He also has let me
> know that his current edition of the software is strictly a 32-bit
> binary compiled for the Sparc V8 processor. I feel that he may get a
> bit better performance if his software vendor provided a binary that
> was tweaked for the V9a but this is where I could be wrong. I have
> also submitted that he should not have spent money on a 8 processor
> capable system which has only two processors in it and then expected
> blistering fast performance. I suggested that a good way to go would
> be a SunFire 280R with dual 1.2GHz processor modules. This could be
> easily upgraded as faster processors became available. I could be
> wrong here but I think that the monolithic CPU/Memory boards in the
> V880 are a tad more expensive and usually a few generations behind the
> CPU modules in the 280R. Perhaps a 64-bit architecture with 1.2GHz
> procs would be quicker than 900MHz procs in a mostly empty V880.

I assume from what you say that he wants to run a specific application
- you talk about his vendor, the fact it's floating point intensive
etc.

If the program is CPU intensive and written as a standard (i.e.
single-threaded) application, then it won't run any faster with 1000
processors than it does with one. If there are more copies of the
program running than there are CPUs, then this does not matter, but if
the number of instances of the program running are less than the
number of CPUs running, then the extra CPUs are of little help. If the
program is multi-threaded it will exploit the multiple CPUs. Some
tasks must inherently be done sequentially, so your can't use multiple
processors to speed things up.

Hence you really need to know if this code is multi-threaded. If not,
you are wasting your time thinking about having lots of CPUs.

One other thing to consider is that a large box full of empty cpu
slots might seem a waste now, but it might soon become very cheap to
fill those cpu slots - at least if CPUs are bought used. The price of
used Sun processors seems to fall quite rapidly. The last 450 MHz
UltrSPARC II processor (X1195A) I fitted in my U80 cost me about 20%
of the price of the first X1195A I bought, only 18 months or so
earlier.



--
Dr. David Kirkby,
Senior Research Fellow,
Department of Medical Physics,
University College London,
11-20 Capper St, London, WC1E 6JA.
Tel: 020 7679 6408 Fax: 020 7679 6269
Internal telephone: ext 46408
e-mail [email]davek@medphys.ucl.ac.uk[/email]

[Rich Teer]

On Thu, 31 Jul 2003, Dr. David Kirkby wrote:

> One reason must surely be that if something goes wrong, it is remotely
> possible to end up with a dead machine. The patch instructions always
> warn of the risks and say the mains must not be removed. There is a
> recovery procedure, but if that fails to work you need to call Sun for
> help.

That is a good point. I still recommend bringing OBPs up to
date (or at least reasonably so), but power loss is perhaps
a cause for concern - or at least, thinking twice.

--
Rich Teer, SCNA, SCSA

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: [url]http://www.rite-online.net[/url]

[Scott Howard]

Dr. David Kirkby <drkirkby@ntlworld.com> wrote:

>> I can't think of a single reason why one WOULDN'T apply OBP
>> patches and the like. I believe that prevention is better
>> than cure.

>
> One reason must surely be that if something goes wrong, it is remotely
> possible to end up with a dead machine. The patch instructions always
> warn of the risks and say the mains must not be removed. There is a
> recovery procedure, but if that fails to work you need to call Sun for
> help.

Most system nowdays have some form of fail-safe method to avoid such
problems.

eg, on the V880's there are actually two copies of the OBP in prom.
When you're doing the upgrade it erases one, re-programs it, and then
verifies it. Only once it's done that does it start on the 2nd copy.

In the event of a non-correctable problem during the upgrade (eg, power
outage halfway through the upgrade) then you'll have one of two
possabilities.

If you were upgrading the "spare" copy of the PROM at the time then the
machine will boot OK (from the "main" copy) and you can just re-run the
upgrade to repair the spare copy.

If you were upgrading the "main" copy of the PROM then you'll need to
move a jumper on the I/O board to boot using the spare copy. You can then
just re-run the upgrade to repair the main copy and (optionally?) move
the jumper back.

For some other system the jump between the two copies happens
automatically if it finds the primary one to be corrupt.

> I'd personally not update the OBP unless the system was on a *decent*
> ups. I've updated the OBP on this U80, but have chosen not to do so on

The U80 also has this two-copy PROM, with a jumper on the motherboard to
jump between the two.

Scott

[Dennis Clarke]

> The price of used Sun processors seems to fall quite rapidly.

Try looking for "cheap" CPU/Memory boards for a V880. Quite a different
matter really than the UltraSparc II modules.

Dennis

[Jay Lessert]

Dennis Clarke <dclarke@blastwave.org> wrote in message news:<Pine.GSO.4.53.0307301039060.18630@blastwave> ...

> I advised him to get a 280R with the fastest UltraSparc III procs that
> money can buy.

Well, that's OK, if 30% faster will change him from very unhappy to
happy. Doesn't sound likely, but you can ask him. Sorta expensive.

BTW, you need to tell us what application he's running, what the VSZ
and RSS are running on a "typical" process, and what the file I/O
patterns of the application are if you want better feedback. We're
all just guessing at this point.

> He says that he needs good floating point speed with
> software written for the UltraSparc processor.

My experience has been that when a user says something like this, it
is seldom true. Even "floating point intensive" apps like Matlab
and Spice spend a lot of time just pushing data around in memory...

> He also has let me
> know that his current edition of the software is strictly a 32-bit
> binary compiled for the Sparc V8 processor. I feel that he may get a
> bit better performance if his software vendor provided a binary that
> was tweaked for the V9a but this is where I could be wrong.

Yeah, probably a little better, but I have the feeling he's looking
for 2X (whether that's possible under any circumstances is another
thing altogether).

> From a storage perspective I had suggested that the Sun StorEdge
> 3510 FC Array would be fairly quick when mated to the X6768A dual
> 2 GB fibre controller.

For single user use, assuming the cache RAM is max'ed (512MB) and
actually turned on, the little ATABOY box is pretty fast. If he has
I/O "problems", it is probably not this boxes fault. The 3510FC is
a fine little box. Again, max the cache and turn it on. But if
you did this upgrade I bet you would get close to zero performance
benefit.

> If money is still an issue then the A5200 with
> FC Hub and dual X6727 with two 1GB ports each would be cheaper and
> allow for four fibre paths to the A5200 array.

I think this would be large step down from the ATABOY, unless his
I/O is of the single-file, gigabyte-range, single-sustained-read/write
type (I bet it's not). For most non-database purposes, a HW-RAID box's
cache RAM is a humongous win.

Easy to find out, of course (assuming his existing disk system is
configured properly). Slap a single 73GB disk in the V880, turn
logging OFF, use 'fastfs fast', and let him use that for a few
benchmark runs.

> I may be wrong here
> but a third party ATA/IDE storage box connected via LVD SCSI3 seems
> somehow "wrong" when mated to a SunFire UltraSparc III class system.

May seem wrong to you, but there's nothing wrong with it in the
application I think you're describing, assuming it's configured
properly.

> The last issue that I got lambasted for was my approach to patches. I
> have been living under the misguided idea that Sun releases patches
> for software because they actually fix issues with the software.

The problem, of course, is that sometimes they also *cause* issues
with "the software". If you've never had that happen, you're very,
very, lucky.

> Applying patches to the OS for things
> like CDE and X11R6 serve no purpose at all.

If you're not using CDE and X11, that is true, of course.

> This is where I think
> I am truely confused. I actually apply patches for things that are
> beyond the scope of the Recommended patch cluster, even on a server
> that will probably never run graphics. This may be a misguided policy
> and should be corrected

If you're spending time on things like this instead of other
duties, or unintentionally breaking applications with unnecessary
patches, perhaps. If not, then who cares?

But there is more to this story than you've told us so far,
I'm guessing. :-)

To summarize:

You've got a user who is an asshole. You sorta need to get
over that, it happens, and CUS can't help except to give you
a place to vent (which is useful, mind you!).

His application doesn't run as fast as he wants. Certainly
part of an admin's job to figure out what the fastest way to
run his app is, and what it would cost. Even if you're
being "asked" six months too late.

Based on little information, but lots of experience, my bottom
line guess is to see if the SW vendor has a Linux port. If
they do, throw him on a $1500 3GHz P4 box. Bet it's at least
1.5X faster.

[Dr. David Kirkby]

Dennis Clarke wrote:

>

> > The price of used Sun processors seems to fall quite rapidly.

>
> Try looking for "cheap" CPU/Memory boards for a V880. Quite a different
> matter really than the UltraSparc II modules.
>
> Dennis

Yes, but the point I'm making is that they eventually do fall one hell
of a lot. Perhaps not this year, but sooner or later they will, and so
a box that was formally full of empty slots might not need too many
$$$ spent to get it running with a full compliment of CPUs.

The UltraSPARC II CPUs are now cheap, but they were 4x their current
price not much over a year ago.

--
Dr. David Kirkby,
Senior Research Fellow,
Department of Medical Physics,
University College London,
11-20 Capper St, London, WC1E 6JA.
Tel: 020 7679 6408 Fax: 020 7679 6269
Internal telephone: ext 46408
e-mail [email]davek@medphys.ucl.ac.uk[/email]

[Philip Brown]

On Wed, 30 Jul 2003 11:34:35 -0400, [email]dclarke@blastwave.org[/email] wrote:

> ...as a V880 with dual 900MHz processors. He also has two X6758A LVD
> SCSI3 adapters in the top two slots of the V880.

it's interesting that you mention this detail. because I believe the V880
is one of those boxes where it matters where you put the PCI cards.
I think it has two separate PCI busses, and "the two top slots" will be
on the same one.

Oops. No, my mistake, it has FOUR. as per page 68 of the owner's guide
[url]http://sunsolve.sun.com/data/806/806-6592/pdf/806-6592-11.pdf[/url]

However, this may not matter speedwise, depending on the answer to below:

> He has then gone and
> connected this to an external third party PC style storage box made by
> NEXSAN. The ATABOY it is called. He hates the performance of the
> V880 processors from a floating point perspective. He hates the perf
> of the external ATABOY. None of this was my idea.

err... is this 2xATABOY, or 1 array, doubly attached?
is it a split array, or is the 2nd connection purely if the first one
fails?


but either way, strictly from a redundancy point of view, they should not
be both in the top two slots.




--
[url]http://www.blastwave.org/[/url] for solaris pre-packaged binaries with pkg-get
Organized by the author of pkg-get
[Trim the no-bots from my address to reply to me by email!]
S.1618 [url]http://thomas.loc.gov/cgi-bin/bdquery/z?d105:SN01618:@@@D[/url]
[url]http://www.spamlaws.com/state/ca1.html[/url]