Professional Web Applications Themes

data manipulation - PERL Miscellaneous

Bob wrote:  http://learn.perl.org/ I'm quite sure that you were able to figure that out yourself, though, and that you actually wanted somebody to write some code, without having given it a try yourself first. :( I'd be surprised if there weren't better ways, but this is one possible approach: #!/usr/bin/perl use strict; use warnings; my ($chunk, chunks); open FH, 'logfile' or die $!; while (<FH>) { if (/^\S/) { push chunks, $chunk if $chunk; $chunk = ''; } $chunk .= $_; } close FH; push chunks, $chunk; # print chunks that include the domain fake.com print grep { /\fake\.com/ } ...

  1. #1

    Default Re: data manipulation

    Bob wrote: 

    http://learn.perl.org/

    I'm quite sure that you were able to figure that out yourself, though,
    and that you actually wanted somebody to write some code, without
    having given it a try yourself first. :( I'd be surprised if there
    weren't better ways, but this is one possible approach:

    #!/usr/bin/perl
    use strict;
    use warnings;

    my ($chunk, chunks);
    open FH, 'logfile' or die $!;
    while (<FH>) {
    if (/^\S/) {
    push chunks, $chunk if $chunk;
    $chunk = '';
    }
    $chunk .= $_;
    }
    close FH;
    push chunks, $chunk;

    # print chunks that include the domain fake.com
    print grep { /\fake\.com/ } chunks;

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl

    Gunnar Guest

  2. #2

    Default Re: data manipulation


    "Gunnar Hjalmarsson" <cc> wrote in message
    news:bm3848$i2lpa$news.uni-berlin.de... 
    >
    > http://learn.perl.org/
    >
    > I'm quite sure that you were able to figure that out yourself, though,
    > and that you actually wanted somebody to write some code, without
    > having given it a try yourself first. :( I'd be surprised if there
    > weren't better ways, but this is one possible approach:
    >
    > #!/usr/bin/perl
    > use strict;
    > use warnings;
    >
    > my ($chunk, chunks);
    > open FH, 'logfile' or die $!;
    > while (<FH>) {
    > if (/^\S/) {
    > push chunks, $chunk if $chunk;
    > $chunk = '';
    > }
    > $chunk .= $_;
    > }
    > close FH;
    > push chunks, $chunk;
    >
    > # print chunks that include the domain fake.com
    > print grep { /\fake\.com/ } chunks;
    >
    > --
    > Gunnar Hjalmarsson
    > Email: http://www.gunnar.cc/cgi-bin/contact.pl
    >[/ref]

    Thanks for the assistance, I was able to use most of what you offered with
    what I already had.

    I am new to Perl, and after reading, there are only 2 lines I don't clearly
    understand

    -> push chunks, $chunk if $chunk;
    -> $chunk = '';

    After reading the push function description from "learning Perl" I am
    failing to understand exactly what is happening here.


    B


    Bob Guest

  3. #3

    Default Re: data manipulation

    Bob <pilbara.net.au> wrote: 


    [ Snip a whole bunch of lines not needed for context. ]

     


    Or, if you like/understand it better written this way:

    if ( $chunk ) {
    push chunks, $chunk;
    }

     


    if $chunk is a true value, then the value of $chunk gets tacked onto
    the end of the chunks array (ie. chunks gets one element larger).

    The second line clears out the aculator for the next go-round.


    --
    Tad McClellan SGML consulting
    com Perl programming
    Fort Worth, Texas
    Tad Guest

  4. #4

    Default Re: data manipulation

    Bob wrote: 

    It adds an element to the array chunks with what's been stored in
    $chunk from previous iterations, and then it empties $chunk.

    The line

    push chunks, $chunk if $chunk;

    can also be written

    if ($chunk) {
    push chunks, $chunk;
    }

    My suggestion means that the whole log file ends up in memory in the
    array chunks. I thought that made the code easier to understand, but
    it should be noted that if the log file is really big, it's not a good
    approach. In that case, you'd better do in the loop with respective
    'generation' of $chunk whatever you want to do with it, and refrain
    from storing the whole file in an array.

    HTH

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl

    Gunnar Guest

  5. #5

    Default Re: data manipulation


    "Gunnar Hjalmarsson" <cc> wrote in message
    news:bm5fel$j9uep$news.uni-berlin.de... 
    >
    > It adds an element to the array chunks with what's been stored in
    > $chunk from previous iterations, and then it empties $chunk.
    >
    > The line
    >
    > push chunks, $chunk if $chunk;
    >
    > can also be written
    >
    > if ($chunk) {
    > push chunks, $chunk;
    > }
    >
    > My suggestion means that the whole log file ends up in memory in the
    > array chunks. I thought that made the code easier to understand, but
    > it should be noted that if the log file is really big, it's not a good
    > approach. In that case, you'd better do in the loop with respective
    > 'generation' of $chunk whatever you want to do with it, and refrain
    > from storing the whole file in an array.
    >
    > HTH
    >
    > --
    > Gunnar Hjalmarsson
    > Email: http://www.gunnar.cc/cgi-bin/contact.pl
    >[/ref]

    Thanks for the explanation, And I already understood the implication of the
    whole file being in memory. In this case, it is not a problem, the log is
    generally ony 3-5Meg but never over 50Meg.

    For those interested, I have included the full script below

    Regards,

    B

    #!/usr/bin/perl
    #
    # NAME qmail-qreadto
    # purpose - look thru the qmail-qread logs for particular messages
    #
    use strict;
    use warnings;


    if (ARGV == "0" ) {
    print "\n\tqmail-qreadto \{ email to search for\} \n";
    print "\t\texample\: qmail-qreadto me\example.net \n\n";
    exit 0
    }

    my ($Log, Logs);
    my $file1 = "/var/log/qmail-qread";
    my $file2 = "/var/log/qmail-qread1";

    if (-e $file2 ) {
    open (MyFILE, $file2);
    } else {
    open (MyFILE, $file1);
    }

    while (<MyFILE>) {
    if (/^\d{1,2}/) {
    push Logs, $Log if $Log;
    $Log = '';
    }
    $Log .= $_;
    }

    close MyFILE;

    push Logs, $Log;

    print grep {/ARGV/} Logs;

    exit 0;



    Bob Guest

  6. #6

    Default Re: data manipulation

    Bob wrote: 

    I'm glad to be able to help, Bob. My apologies for being assumptive in
    my first reply - you were obviously ready to put more effort in it
    than I first thought. ;-)
     

    Looks good to me.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl

    Gunnar Guest

  7. #7

    Default Re: data manipulation

    Bob <pilbara.net.au> wrote:


    [ snip yet another full-quote. Please learn to quote followups properly ]

     


    Why use a numeric operator and then force stringification?

    If you want to test a number, use a number and a numeric operator:

    if (ARGV == 0 ) {

    If you want to test a string, use a string and a string operator

    if (ARGV eq "0" ) {

     
    ^ ^
    ^ ^

    A useless use of backslashes.

     


    You should exit with a *non* zero value when an error occurs.

     


    You should always, yes *always*, check the return value from open():

    open(MyFILE, $file2) or die "could not open '$file2' $!";


    --
    Tad McClellan SGML consulting
    com Perl programming
    Fort Worth, Texas
    Tad Guest

  8. #8

    Default Re: data manipulation


    "Tad McClellan" <com> wrote in message
    news:augustmail.com... 
    >
    >
    > Why use a numeric operator and then force stringification?
    >
    > If you want to test a number, use a number and a numeric operator:
    >
    > if (ARGV == 0 ) {
    >
    > If you want to test a string, use a string and a string operator
    >
    > if (ARGV eq "0" ) {[/ref]


    Ok, what I am trying to test for is the existance of an argument, - if
    ARGV is NULL .... do something..... what I use works, but if there is a
    better way of testing for a NULL than a numerical comparison? am open to
    suggestions.... In this case, where the array is empty, a numeric test
    returns "0", so the test made sense.
     
    > ^ ^
    > ^ ^
    >
    > A useless use of backslashes.
    >
    >[/ref]

    only quoted as when I was testing a warning was generated, but I later found
    the real problem.....
     
    >
    >
    > You should exit with a *non* zero value when an error occurs.
    >
    >[/ref]

    I am exiting without an error here , simply if() do something exit -this
    is just as valid an exit from the code as any other exit, and no error has
    occurred.
     
    >
    >
    > You should always, yes *always*, check the return value from open():
    >
    > open(MyFILE, $file2) or die "could not open '$file2' $!";
    >[/ref]

    In this case, I know that 1 of the 2 files will always exist, so I did not
    "or die" here, but for the sake of consistency I will add it.


    I appreciate the feedback given, and suggestions made.

    Bob



    Bob Guest

  9. #9

    Default Re: data manipulation

    "Bob" <pilbara.net.au> wrote in message news:<norcom.net.au>... 
    > >
    > > Why use a numeric operator and then force stringification?
    > >
    > > If you want to test a number, use a number and a numeric operator:
    > >
    > > if (ARGV == 0 ) {
    > >
    > > If you want to test a string, use a string and a string operator
    > >
    > > if (ARGV eq "0" ) {[/ref]
    >
    >
    > Ok, what I am trying to test for is the existance of an argument, - if
    > ARGV is NULL .... do something..... what I use works, but if there is a
    > better way of testing for a NULL than a numerical comparison?[/ref]

    Yep, simple boolean context.

    unless (ARGV) {
     

    No, I cannot see any reason why it never makes sense to use a string
    literal in a numeric context. (Of course the compiler optomises the
    string-to-number convertions out).
     
    > >
    > >
    > > You should always, yes *always*, check the return value from open():
    > >
    > > open(MyFILE, $file2) or die "could not open '$file2' $!";
    > >[/ref]
    >
    > In this case, I know that 1 of the 2 files will always exist, so I did not
    > "or die" here, but for the sake of consistency I will add it.[/ref]

    No it's not just for consistency. It's a good programming habit so
    that if something sometime in the future breaks you get a useful
    error.
    nobull@mail.com Guest

  10. #10

    Default Re: data manipulation

    Bob wrote: 

    That is usually written as:

    unless ( ARGV ) {

     

    What happens if the file is unlinked or moved between the time you stat
    the file and the time you open the file? What happens if neither $file1
    nor $file2 exists? This is better:

    open MyFILE, $file2
    or open MyFILE, $file1
    or die "Cannot open $file2 or $file1: $!";

     

    The match operator m// interpolates its contents just like a double
    quoted string so if you have more then one element in ARGV this will
    not work. You probably want:

    print grep /\Q$ARGV[0]/, Logs;

    Or you could test the contents of $Log before you push it into Logs:

    push Logs, $Log if $Log =~ /\Q$ARGV[0]/;

     


    John
    --
    use Perl;
    program
    fulfillment
    John Guest

Similar Threads

  1. Data Manipulation Techniques
    By MeltdownMX in forum Coldfusion - Advanced Techniques
    Replies: 3
    Last Post: March 10th, 01:32 PM
  2. Old ASP - data manipulation
    By Oded in forum ASP Database
    Replies: 1
    Last Post: June 22nd, 05:39 AM
  3. Ini File Manipulation
    By Alexander Popkov in forum ASP Components
    Replies: 2
    Last Post: May 3rd, 01:59 PM
  4. pdf manipulation
    By Chris in forum PHP Development
    Replies: 8
    Last Post: December 19th, 06:04 AM
  5. data manipulation
    By Michael Myett in forum FileMaker
    Replies: 3
    Last Post: October 26th, 06:55 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139