Professional Web Applications Themes

sum a column - PERL Beginners

This is a beginner’s mailing list. I’m sorry my script is not at your level, I'm learning. I think telling someone on a beginners mailing list that they don’thave a clue is inappropriate. It shows me you have no clue. -----Original Message----- From: Gunnar Hjalmarsson [mailto:cc] Sent: Friday, October 01, 2004 2:53 AM To: org Subject: Re: sum a column Rmck wrote:  Indeed.   That awkward attempt seems to be made by someone who hasn't a clue about programming.   The script author does have a clue about programming. -- Gunnar Hjalmarsson Email: http://www.gunnar.cc/cgi-bin/contact.pl -- To unsubscribe, e-mail: org For ...

  1. #1

    Default Re: sum a column

    This is a beginner’s mailing list. I’m sorry my script is not at your level, I'm learning.
    I think telling someone on a beginners mailing list that they don’thave a clue is inappropriate.
    It shows me you have no clue.


    -----Original Message-----
    From: Gunnar Hjalmarsson [mailto:cc]
    Sent: Friday, October 01, 2004 2:53 AM
    To: org
    Subject: Re: sum a column

    Rmck wrote: 

    Indeed.
     

    That awkward attempt seems to be made by someone who hasn't a clue
    about programming.
     

    The script author does have a clue about programming.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl

    --
    To unsubscribe, e-mail: org
    For additional commands, e-mail: org
    <http://learn.perl.org/> <http://learn.perl.org/first-response>

    -----Original Message-----
    From: Gunnar Hjalmarsson [mailto:cc]
    Sent: Friday, October 01, 2004 2:53 AM
    To: org
    Subject: Re: sum a column

    Rmck wrote: 

    Indeed.
     

    That awkward attempt seems to be made by someone who hasn't a clue
    about programming.
     

    The script author does have a clue about programming.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl

    --
    To unsubscribe, e-mail: org
    For additional commands, e-mail: org
    <http://learn.perl.org/> <http://learn.perl.org/first-response>

    Rmck Guest

  2. #2

    Default Re: sum a column

    I tried the suggestion below, and it appears to have worked, thanks.

    I had to add a "print $sum !=" to inside and outside the while loop. Im not sure why it is working this way?? Also Im getting an error "Use of uninitialized value in string ne at ./clean1.pl line 28, <LOG> line 1." ??
    If anyone can explain I would appericate:


    #!/usr/bin/perl
    use Socket;
    use strict;
    use POSIX 'strftime';
    use warnings;
    my $line = $ARGV[0];

    my $time = strftime "%y%m%d%H", localtime;
    my $sum = 0;
    my $prev_date;

    # open the file
    open(LOG,"$line") or die "Unable to open LOG:$!\n";
    print "Date_Time, SRCIP, DSTIP, TOTALBYTES \n";

    # read it in one record at a time
    #while (<LOG>) {
    while ($line = <LOG>) {
    next if $line =~ /^\D/;
    my ($logdate,$srcip,$dstip,$totalbytes) = split(/\t/,$line);
    my ($date,$time)= split(/\s/,$logdate);
    my hour = split(/:/,$time);
    my $current_date = $date;


    if ($hour[0] >= 6 and $hour[0] < 22){
    if ($prev_date ne $current_date) { ##LINE 28##
    #print "Total: $sum\n"; # display before clearing, Prints a Total: 0 on first line
    if ($sum != 0) {
    print "Total: $sum\n";
    }
    $sum = 0;
    }
    $sum += $totalbytes;
    $prev_date = $current_date;

    print "$logdate,$srcip,$dstip,$totalbytes";
    }

    # End Of While:
    }
    if ($sum != 0) {
    print "Total: $sum\n";
    }

    # close the file
    close(LOG);

    output:
    Date_Time, SRCIP, DSTIP, TOTALBYTES
    Use of uninitialized value in string ne at ./clean1.pl line 28, <LOG> line 1.
    01-01-2004 12:56:48,192.168.1.1,192.168.2.2,2768
    01-01-2004 12:56:48,192.168.2.2,192.168.1.1,438
    Total: 3206
    01-02-2004 16:49:45,192.168.3.3,192.168.4.4,364
    01-02-2004 16:49:45,192.168.4.4,192.168.3.3,513
    Total: 877


    Thank You.
    Rob




    -----Original Message-----
    From: Wiggins d Anconia [mailto:org]
    Sent: Thursday, September 30, 2004 5:32 PM
    To: rmck; org
    Subject: Re: sum a column
     

    Hash shouldn't be necessary unless they are not ordered.
     

    No need to quote integers during assignment.
     

    Why are you opening ARGV, doesn't seem like this would work, that or
    Perl is doing something under the hood that I don't expect, but very
    little surprises me :-).
     

    You can capture the above to get just the hour if you want to, with,

    my ($hour) = split(/:/, $time);
     

    Why do this down here, it seems it would be more efficient to catch it
    as early as possible.
     

    This is likely your problem. Are you really trying to sum the # of
    bytes? If so why are you multiplying it times the $count, which is
    seemingly the line of the file that you are currently processing. Also,
    $sum will have to be scoped ouside the loop otherwise it will be reset
    for each line of input that you process, you really need to keep it for
    each iteration and then only clear it when the hour/date you are
    currently on is not the same as the previous hour/date.
     

    So in pseudo code it will look something like:

    my $sum = 0;
    my $prev_date;
    while (my $line = <LOG>) {
    Rmck Guest

  3. #3

    Default sum a column

    Hello,

    Im trying to sum up a column from my results. Help.

    current output:
    Date_Time, SRCIP, DSTIP, TOTALBYTES
    01-01-2004 12:56:48, 192.168.1.1, 192.168.2.2, 2768
    Sum Of Bytes = 2768
    01-01-2004 12:56:48, 192.168.2.2, 192.168.1.1, 438
    Sum Of Bytes = 876
    01-02-2004 16:49:45, 192.168.3.3, 192.168.4.4, 9058
    Sum Of Bytes = 27174
    01-02-2004 16:49:45, 192.168.4.4, 192.168.3.3, 918
    Sum Of Bytes = 3672

    goal:
    Date_Time, SRCIP, DSTIP, TOTALBYTES
    01-01-2004 12:56:48, 192.168.1.1, 192.168.2.2, 2768
    01-01-2004 12:56:48, 192.168.2.2, 192.168.1.1, 438
    Sum Of Bytes = 3206
    01-02-2004 16:49:45, 192.168.3.3, 192.168.4.4, 364
    01-02-2004 16:49:45, 192.168.4.4, 192.168.3.3, 513
    Sum Of Bytes = 877


    Im stuck. Should I use a hash??

    Current Script:
    #!/usr/bin/perl
    use Socket;
    use strict;
    use POSIX 'strftime';
    use warnings;
    my $time = strftime "%y%m%d%H", localtime;
    my $count = "0";

    # open the file
    open(LOG,"ARGV") or die "Unable to open LOG:$!\n";

    print "Date_Time, SRCIP, DSTIP, TOTALBYTES \n";

    # read it in one record at a time
    while (<LOG>) {
    my ($logdate,$srcip,$dstip,$totalbytes) = split(/\t/,$_);
    my ($date,$time )= split(/\s/,$logdate);
    my hour = split(/:/,$time);

    next if $_ =~ /^\D/;

    if ($hour[0] >= 6 and $hour[0] < 22){
    print "$logdate,$srcip,$dstip,$totalbytes";
    $count++;
    my $sum = $count * $totalbytes;
    print "Sum of Bytes = $sum\n";
    }
    }
    # close the file
    close(LOG);


    Thanks for any input......

    Rob

    Rmck Guest

  4. #4

    Default Re: sum a column

    Rmck wrote: 

    <job specification snipped>
     

    I don't believe you. To be stuck, you need to try first, and I suspect
    that you didn't do that. Note that this is not a free service for
    modifying scripts that people pick up somewhere.

    I suggest that you make a serious attempt to solve the 'problem', and
    come back here if you encounter problems.

    If you don't know any Perl at all, start here:

    http://learn.perl.org/

    or hire a consultant to help you.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Guest

  5. #5

    Default Re: sum a column

    Are you serious..... Im not sure how to sum up the column... I tried by using the field and * it by the increment.

    I have written every part of my script, Im stuck on one part , summing a column....

    Man you need a vacation.... Gunnar should be Goner.......



    -----Original Message-----
    From: Gunnar Hjalmarsson <cc>
    Sent: Sep 30, 2004 5:21 PM
    To: org
    Subject: Re: sum a column

    Rmck wrote: 

    <job specification snipped>
     

    I don't believe you. To be stuck, you need to try first, and I suspect
    that you didn't do that. Note that this is not a free service for
    modifying scripts that people pick up somewhere.

    I suggest that you make a serious attempt to solve the 'problem', and
    come back here if you encounter problems.

    If you don't know any Perl at all, start here:

    http://learn.perl.org/

    or hire a consultant to help you.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl

    --
    To unsubscribe, e-mail: org
    For additional commands, e-mail: org
    <http://learn.perl.org/> <http://learn.perl.org/first-response>



    Rmck Guest

  6. #6

    Default Re: sum a column

    > Hello, 

    Hash shouldn't be necessary unless they are not ordered.
     

    No need to quote integers during assignment.
     

    Why are you opening ARGV, doesn't seem like this would work, that or
    Perl is doing something under the hood that I don't expect, but very
    little surprises me :-).
     

    You can capture the above to get just the hour if you want to, with,

    my ($hour) = split(/:/, $time);
     

    Why do this down here, it seems it would be more efficient to catch it
    as early as possible.
     

    This is likely your problem. Are you really trying to sum the # of
    bytes? If so why are you multiplying it times the $count, which is
    seemingly the line of the file that you are currently processing. Also,
    $sum will have to be scoped ouside the loop otherwise it will be reset
    for each line of input that you process, you really need to keep it for
    each iteration and then only clear it when the hour/date you are
    currently on is not the same as the previous hour/date.
     

    So in pseudo code it will look something like:

    my $sum = 0;
    my $prev_date;
    while (my $line = <LOG>) {
    Wiggins Guest

  7. #7

    Default Re: sum a column

    Rmck wrote: 

    Indeed.
     

    That awkward attempt seems to be made by someone who hasn't a clue
    about programming.
     

    The script author does have a clue about programming.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Guest

  8. #8

    Default Re: sum a column

    Rmck wrote: 
    >>
    >> That awkward attempt seems to be made by someone who hasn't a
    >> clue about programming.
    >> 
    >>
    >> The script author does have a clue about programming.[/ref]
    >
    > This is a beginner's mailing list. I'm sorry my script is not at
    > your level, I'm learning.[/ref]

    Please show us that you are learning! Prove that I'm wrong by showing
    us how you applied Wiggins' helpful advice in your code.
     

    But I didn't say that about the script author, and you claim to be the
    script author, so what's the problem?

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Guest

  9. #9

    Default RE: sum a column




    looks like a communications problem...

    "The script author does have a clue about programming."

    is easy to misread as saying "does not" if you pass your eyes over it
    quickly... i did that the first time i saw it... *shrug*





    willy
    http://www.hackswell.com/corenth

    William Guest

  10. #10

    Default RE: sum a column

    rmck <net> wrote:

    Please stop top posting.

    : I had to add a "print $sum !=" to inside and outside the
    : while loop. Im not sure why it is working this way?? Also
    : Im getting an error "Use of uninitialized value in string
    : ne at ./clean1.pl line 28, <LOG> line 1." ??

    Break it down to a smaller test case.

    use strict;
    use warnings;

    my $prev_date;
    if ( $prev_date ne 'foo' ) { ## LINE 28 ##
    print 'foo';
    }

    Since prev_date does not have value, it cannot be compared
    without raising a warning. On the second pass $prev_date has
    a value.


    : #!/usr/bin/perl
    : use Socket;
    : use strict;
    : use POSIX 'strftime';
    : use warnings;
    : my $line = $ARGV[0];

    Better named $file or $log_file.


    : my $time = strftime "%y%m%d%H", localtime;

    Not used in this script.


    : my $sum = 0;
    : my $prev_date;

    Undefined on first pass of the while loop.


    : # open the file
    : open(LOG,"$line") or die "Unable to open LOG:$!\n";

    Don't quote $line. Don't place a newline after $!.
    It suppresses line number info.

    open LOG, $line or die "Unable to open LOG: $!";

    Or:

    open LOG, "<$line" or die "Unable to open LOG: $!";


    : print "Date_Time, SRCIP, DSTIP, TOTALBYTES \n";

    Ahem...

    print "Date_Time, SRCIP, DSTIP, TOTALBYTES\n";


    : # read it in one record at a time
    : #while (<LOG>) {
    : while ($line = <LOG>) {

    while ( my $line = <LOG> ) {

    : next if $line =~ /^\D/;

    chomp $line;

    : my ($logdate,$srcip,$dstip,$totalbytes) = split(/\t/,$line);
    : my ($date,$time)= split(/\s/,$logdate);

    Why do $current_date amd $prev_date use the underscore
    to separate words and the other variables use abbreviations
    and no seperation?

    my( $log_date,
    $source_ip,
    $destination_ip,
    $total_bytes, ) = split /\t/, $line;


    : my hour = split(/:/,$time);

    Better written as a scalar.

    my $hour = ( split /:/, $time )[0];


    : my $current_date = $date;

    : if ($hour[0] >= 6 and $hour[0] < 22){
    : if ($prev_date ne $current_date) { ##LINE 28##

    $prev_date is not defined on first pass.


    : #print "Total: $sum\n"; # display before clearing,
    : Prints a Total: 0 on first line
    : if ($sum != 0) {
    : print "Total: $sum\n";
    : }
    : $sum = 0;
    : }
    : $sum += $totalbytes;
    : $prev_date = $current_date;
    :
    : print "$logdate,$srcip,$dstip,$totalbytes";

    You probably want a newline at the end.

    print "$logdate,$srcip,$dstip,$totalbytes\n";


    : }
    :
    : # End Of While:
    : }


    You don't need this if you line up your indentation.
    Every time you start a new code block indent your code.
    When the block ends, outdent it.

    while ( <LOG> ) {
    next if /^\D/;

    chomp;

    my( $log_date,
    $source_ip,
    $destination_ip,
    $total_bytes ) = split /\t/;

    my( $current_date, $time ) = split ' ', $log_date;

    my $hour = ( split /:/, $time )[0];

    if ( $hour > 5 and $hour < 22 ) {
    if ( $prev_date ne $current_date ) {
    print "Total: $sum\n" if $sum;

    $prev_date = $current_date;
    $sum = 0;
    }
    $sum += $total_bytes;

    print "$log_date,$source_ip,$destination_ip,$total_bytes ";
    }
    }


    HTH,

    Charles K. Clarkson
    --
    Mobile Homes Specialist
    254 968-8328




    Charles Guest

  11. #11

    Default RE: sum a column


    --- "Charles K. Clarkson" <net>
    wrote:

    [snip] 

    Isn't $prev_date assigned to '', and isn't that
    different than being undefined? I guess I'm confused
    on that.

    [snip]
     

    I've always wondered why it worked sometimes and
    didn't work others. Thanks fo the info.

    Sorry about the way yahoo quotes stuff. Need to find
    another web based email program.

    Denzil




    __________________________________
    Do you Yahoo!?
    Yahoo! Mail Address AutoComplete - You start. We finish.
    http://promotions./new_mail
    Denzil Guest

  12. #12

    Default Re: sum a column

    >>my $prev_date; 
    >
    >
    > Isn't $prev_date assigned to '', and isn't that[/ref]

    It is if you do
    my $prev_date = '';
    but
    my $prev_date;
    it is "uninitialized" as in it hasn't been assigned any value incuding
    "empty"

    Compare
    perl -mstrict -we 'my $v;for(1..3) { print "$v\n";$v++; }'
    with
    perl -mstrict -we 'my $v = "";for(1..3) { print "$v\n";$v++; }'

    Its kind of like the difference in SQL with an empty value and a NULL value.

    HTH :)

    Lee.M - JupiterHost.Net
    JupiterHost.Net Guest

  13. #13

    Default Re: sum a column


    --- "JupiterHost.Net" <net> wrote:
     
    >
    > It is if you do
    > my $prev_date = '';
    > but
    > my $prev_date;
    > it is "uninitialized" as in it hasn't been assigned
    > any value incuding
    > "empty"
    >
    > Compare
    > perl -mstrict -we 'my $v;for(1..3) { print
    > "$v\n";$v++; }'
    > with
    > perl -mstrict -we 'my $v = "";for(1..3) { print
    > "$v\n";$v++; }'
    >[/ref]

    Hmm, I could swear I read in my manual that perl will
    automatically initialize variables. But, that's
    obviously not case, as your example demonstrates.

    Thanks for the info!

    Denzil



    _______________________________
    Do you Yahoo!?
    Declare Yourself - Register online to vote today!
    http://vote.
    Denzil Guest

  14. #14

    Default Re: sum a column

    rmck wrote: 

    Hello,
     

    It looks like a hash should help.

     

    Why are you assigning a string to $count when it will be used in a numerical
    context? Although perl will do the right thing and treat it as the number
    zero, someone else reading your code might not understand it.

    my $count = 0;

     

    That is the same as saying:

    open(LOG,join($",ARGV)) or die "Unable to open LOG:$!\n";

    Which will *ONLY* work if there is *ONE* file name on the command line.


    open(LOG,$ARGV[0]) or die "Unable to open $ARGV[0]:$!\n";

     

    You should run this test first so you don't try to split an invalid line.

     


    Since you didn't provide an example of the input data I can't test this but
    this should be close to what you want:


    #!/usr/bin/perl
    use warnings;
    use strict;

    use Socket;
    use POSIX 'strftime';
    my $time = strftime '%y%m%d%H', localtime;


    print "Date_Time, SRCIP, DSTIP, TOTALBYTES\n";

    my %data;
    # read it in one record at a time
    while ( <> ) {
    next unless /^(\d\d-\d\d-\d{4}
    [ ]
    (\d\d):\d\d:\d\d)
    \t
    (\d{1,3}(?:\.\d{1,3}){3})
    \t
    (\d{1,3}(?:\.\d{1,3}){3})
    \t
    (\d+)
    \n
    /x;
    next if $2 < 6 or $2 >= 22;
    my ( $date, $srcip, $dstip, $bytes ) = ( $1, $3, $4, $5 );

    if ( exists $data{ $date } ) {
    push { $data{ $date }{ lines } }, "$date,$srcip,$dstip,$bytes\n";
    $data{ $date }{ total } += $bytes;
    }
    else {
    print { $data{ $date }{ lines } }, "Sum of Bytes =
    $data{$date}{total}\n";
    $data{ $date }{ lines } = [ "$date,$srcip,$dstip,$bytes\n" ];
    $data{ $date }{ total } = $bytes;
    }
    }



    John
    --
    use Perl;
    program
    fulfillment
    John Guest

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139