Professional Web Applications Themes

using substr... is this efficient - PERL Beginners

I have a scripted I wrote that uses *substr* on a file that has fixed fields. I have to p the file before I upload into MYSQL. Is there a more efficient way other than *substr*, I looked a *pack* and *unpack* . it did not make any sense to me. Here is my code, and any suggestions, comments would be appreciated!!! #----------------------------------------------- #!/usr/bin/perl -w # Sample input file # 304 THE CASTLE TWO DRAWER CHEST 6-168 02/18/04 # 751 FRUIT STILL LIFE PLANTER 6-22 02/18/04 # 1029 GLASS TOP: 24" DIA X 1/2" ---- 02/18/04 $pvr = 'DISCSL.txt'; $out ...

  1. #1

    Default using substr... is this efficient

    I have a scripted I wrote that uses *substr* on a file that has fixed
    fields. I have to p the file before I upload into MYSQL.



    Is there a more efficient way other than *substr*, I looked a *pack* and
    *unpack* . it did not make any sense to me.



    Here is my code, and any suggestions, comments would be appreciated!!!

    #-----------------------------------------------



    #!/usr/bin/perl -w

    # Sample input file

    # 304 THE CASTLE TWO DRAWER CHEST 6-168 02/18/04

    # 751 FRUIT STILL LIFE PLANTER 6-22 02/18/04

    # 1029 GLASS TOP: 24" DIA X 1/2" ---- 02/18/04



    $pvr = 'DISCSL.txt';

    $out = 'DISCSL_out.txt';



    open FILE, "<$pvr" or die "Unabel to open the file\n";

    open DATA, ">$out" or die "Unabel to open the file\n";



    while(<FILE>)

    {

    $the_line = $_;

    chomp($the_line);



    $item = substr($the_line,0,10);

    $ldesc = substr($the_line,11,40);

    $page = substr($the_line,52,6);

    $d = substr($the_line,59,8);



    # Remove all spaces we do not need !!!

    $item =~ s/\s+// ;

    $page =~ s/\s+// ;



    if ( $page eq "----" ) { $page = "" };

    print DATA "$item|$ldesc|$page|$d\n";



    }



    close FILE;

    close DATA;



    exit;



    Larry Sandwick

    Sarreid, Ltd.

    Network/System Administrator

    phone: (252) 291-1414 x223

    fax : (252) 237-1592




    Larry Guest

  2. #2

    Default RE: using substr... is this efficient

    > > Is there a more efficient way other than *substr*, I looked a *pack* and [/ref]

    Unpack is what I would use. Perldoc unpack.

    $item = substr($the_line,0,10);
    $ldesc = substr($the_line,11,40);
    $page = substr($the_line,52,6);
    $d = substr($the_line,59,8);

    my ( $time, $ldesc, $page, $d ) = unpack ( "A10 A40 A6 A8", $the_line );


    untested.

    HTH,
    PK

    Paul Guest

  3. #3

    Default Re: using substr... is this efficient

    Larry Sandwick wrote: 

    Hi Larry.

    You need to use

    my ($item, $ldesc, $page, $d) = unpack 'A10xA40xA6xA8', $the_line;

    but I make the $ldesc field 28 characters long instead of the 40 you have
    codede, so perhaps you need a format of 'A10xA28xA6xA8'?

    The 'A' unpack format strips trailing spaces from the input string so you
    don't need to adjust the $page value afterwards. One the other hand, if
    you want the fields exactly as they are, spaces and all, then use a lower
    case 'a' format instead.

    I hope this helps,

    Rob



    Rob Guest

  4. #4

    Default RE: using substr... is this efficient

    > -----Original Message----- 

    From my own limited performance testing (using the Benchmark module),
    I've found that substr is faster than unpack.

    Luke
    Luke Guest

  5. #5

    Default Re: using substr... is this efficient

    Luke Bakken wrote: 
    >
    > From my own limited performance testing (using the Benchmark module),
    > I've found that substr is faster than unpack.[/ref]

    Thanks for that Luke. It seems you're right:

    use Benchmark;

    my $the_line = ' 304 THE CASTLE TWO DRAWER CHEST 6-168 02/18/04';

    timethese(1E7, {
    'unpack' => sub {
    my ($item, $ldesc, $page, $d) = unpack 'A10xA28xA6xA8', $the_line;
    },
    'substr' => sub {
    my $item = substr($the_line, 0, 10);
    my $ldesc = substr($the_line, 11, 28);
    my $page = substr($the_line, 40, 6);
    my $d = substr($the_line, 47, 8);
    },
    });

    **OUTPUT

    Benchmark: timing 10000000 iterations of substr, unpack...
    substr: 19 wallclock secs (18.19 usr + 0.00 sys = 18.19 CPU) 549752.61/s (n=10000000)
    unpack: 31 wallclock secs (30.64 usr + 0.00 sys = 30.64 CPU) 326370.76/s (n=10000000)

    I think that will surpirise quite a few people.

    Rob


    Rob Guest

Similar Threads

  1. #40754 [NEW]: substr() checks overflow
    By christopher dot jones at oracle dot com in forum PHP Bugs
    Replies: 2
    Last Post: March 9th, 02:00 AM
  2. Help translating fread to substr
    By RootShell in forum PHP Development
    Replies: 1
    Last Post: May 4th, 10:02 PM
  3. substr parsing mask
    By perl@swanmail.com in forum PERL Beginners
    Replies: 4
    Last Post: September 29th, 03:11 PM
  4. [PHP] substr ?
    By John Taylor-Johnston in forum PHP Development
    Replies: 0
    Last Post: August 17th, 05:49 AM
  5. InStr, substr, Mid ???
    By J. Muenchbourg in forum ASP
    Replies: 1
    Last Post: August 13th, 12:18 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139