Benchmark's cmpthese results - how to read

[Kevin Old]

Hello everyone,

I have a two sets of code that I'd like to know which one is "fastest",
so I turned to the Benchmark module to help solve the mystery. My code:

#!/usr/bin/perl

use warnings;
use strict;
use Benchmark qw( timethese cmpthese);

my $string = '01/23/2004';
my $x = 3;

my $r = timethese( -5, {
sbstr => sub{my $ko = substr($string, 0, 2)},
regex => sub{my $ko = $string =~ /(\d{2})/},
} );

cmpthese $r;

Returns:

Benchmark: running regex, sbstr for at least 5 CPU seconds...
regex: 4 wallclock secs ( 5.00 usr + 0.00 sys = 5.00 CPU) @
309749.20/s (n=1548746)
sbstr: 5 wallclock secs ( 5.28 usr + 0.00 sys = 5.28 CPU) @
993059.28/s (n=5243353)
Rate regex sbstr
regex 309749/s -- -69%
sbstr 993059/s 221% --

This is great information, but a little confusing. After looking over
the POD documentation for Benchmark, I think I understand, but would
like clarification.

If I understand correctly, of the 5 CPU seconds each subroutine was run
the 'regex' subroutine took 4 wallclock seconds and the 'sbstr'
subroutine took 5 wallclock seconds.

What's confusing is everything after the word "seconds" for each
subroutine...and the chart that has percentages in it. Does that chart
mean that the 'regex' subroutine is 221% "faster" than the 'sbstr'
subroutine?

Also, what is the "translation" of '( 5.00 usr + 0.00 sys = 5.00 CPU)
@ 309749.20/s (n=1548746)' and '( 5.28 usr + 0.00 sys = 5.28 CPU) @
993059.28/s (n=5243353)'. I assume from the percentage chart that the
'309749.20/s' and '993059.28/s' is the Rate, but what does that actually
mean?

Any help is appreciated,
Kevin

--
Kevin Old <kold@kold.homelinux.com>

[Kenton Brede]

On Sun, Jan 25, 2004 at 01:59:30PM -0500, Kevin Old (kold@kold.homelinux.com) wrote:

> Hello everyone,
>
> I have a two sets of code that I'd like to know which one is "fastest",
> so I turned to the Benchmark module to help solve the mystery. My code:
>
> #!/usr/bin/perl
>
> use warnings;
> use strict;
> use Benchmark qw( timethese cmpthese);
>
> my $string = '01/23/2004';
> my $x = 3;
>
> my $r = timethese( -5, {
> sbstr => sub{my $ko = substr($string, 0, 2)},
> regex => sub{my $ko = $string =~ /(\d{2})/},
> } );
>
> cmpthese $r;
>
> Returns:
>
> Benchmark: running regex, sbstr for at least 5 CPU seconds...
> regex: 4 wallclock secs ( 5.00 usr + 0.00 sys = 5.00 CPU) @
> 309749.20/s (n=1548746)
> sbstr: 5 wallclock secs ( 5.28 usr + 0.00 sys = 5.28 CPU) @
> 993059.28/s (n=5243353)
> Rate regex sbstr
> regex 309749/s -- -69%
> sbstr 993059/s 221% --
>
> This is great information, but a little confusing. After looking over
> the POD documentation for Benchmark, I think I understand, but would
> like clarification.
>
> If I understand correctly, of the 5 CPU seconds each subroutine was run
> the 'regex' subroutine took 4 wallclock seconds and the 'sbstr'
> subroutine took 5 wallclock seconds.
>

Yes, although from what limited experience I have with the module, when
your using "CPU seconds" the two "wall clock" figures will be very
similar. Now if you change -

"my $r = timethese( -5, {"
to -
"my $r = timethese( 1000, {"

In other words do 1000 iterations of each subroutine. Then I think you
will see a greater variance between "wall times."

> What's confusing is everything after the word "seconds" for each
> subroutine...and the chart that has percentages in it. Does that chart
> mean that the 'regex' subroutine is 221% "faster" than the 'sbstr'
> subroutine?
>

No, sbstr is 221% faster than regex
regex is 69% slower than sbstr

> Also, what is the "translation" of '( 5.00 usr + 0.00 sys = 5.00 CPU)
> @ 309749.20/s (n=1548746)' and '( 5.28 usr + 0.00 sys = 5.28 CPU) @
> 993059.28/s (n=5243353)'. I assume from the percentage chart that the
> '309749.20/s' and '993059.28/s' is the Rate, but what does that actually
> mean?

"5.00 usr" refers to CPU percentage in user mode. "0.00 sys" refers to
CPU percentage in system mode. Add them together and you get '5.00% total
CPU time' for the first and 5.28% for the second.

"993059.28/s" I think is the number of iterations the subroutine
performed in a second. I'm not sure what "(n=5243353)" refers to for
sure but it might be the actual number of iterations.

Hope I got all that right. I'm sure someone will clear it up if I
don't.
Kent

--