Which is better, opening and closing, or a massive file opening and a massive closing?

[Lonewolf@Nc.Rr.Com]

I'm still working on the script, though it is cleaning up even better, I
even have it running cleaner by dumping anything not matching some specs.
What I am trying to figure out though is this:

I have 42 places for the output to go, 1 of those is a constant dump, the
others are all based on whether or not there is data in a field. If the
data is in the field, the data writes to a file with the same name as the
data checked. If not then it writes to a global catch-all.

<!-- snip -->

open OUTFILE, ">/home/multifax/everyone" or die "Can't open ${infile}.out
at home: $!";
open OUTFILE1, ">/home/multifax/102" or die "Can't open ${infile}.out at
home: $!";

print OUTFILE "$fields[1]\@$tmptxt\n";

if ($fields[11] eq 102){
print OUTFILE1 "$fields[1]\@$tmptxt\n";
}

<!-- snap -->

I am wondering if it is more processor intensive to open the 42 separate
files at one time, parse the data, and then close all the files, or if I
should try to re-write the parse to open the correct file, dump the data,
and then close that file, then repeat the process. I know programmatically
it is probably better to open and close the files as there would be no more
copy and pasting, but was thinking processor intensive. As it is right now
it takes the script about 5 seconds to parse the 557 lines of data.

If done in a loop, would it look something like this???:

<!-- snip -->
@filees = ('102','104',118');

if (grep $fields[4] eq $_, @filees) {
open OUTFILE1, ">/home/multifax/$_" or die "Can't open $_ !";
print OUTFILE1 "$fields[1]\@$tmptxt\n";
close OUTFILE1;
}

<!-- snap -->


I'm trying to apply what I have learned from you guys and from breaking my
code in the last few days in every script I am writing.

Thanks,
Robert

--------------------------------------------------------------------
mail2web - Check your email from the web at
[url]http://mail2web.com/[/url] .

[Jenda Krynicky]

From: "LoneWolf@nc.rr.com" <LoneWolf@nc.rr.com>

> I am wondering if it is more processor intensive to open the 42
> separate files at one time, parse the data, and then close all the
> files, or if I should try to re-write the parse to open the correct
> file, dump the data, and then close that file, then repeat the
> process. I know programmatically it is probably better to open and
> close the files as there would be no more copy and pasting, but was
> thinking processor intensive. As it is right now it takes the script
> about 5 seconds to parse the 557 lines of data.

If you have this small data it would be best to in the memory, into
42 separate strings (in an array or hash of course!) and then at the
end loop through them and flush their contents to the files.

If you expect more data in the future you'd better open all 42 files
in the beginning, put their handles into an array or hash, then print
the lines as you go and close all files at the end.

Otherwise you spend most of the time opening and closing files. IMHO
of course.


something like

my @filees = ('102','104',118');

my %handles;
foreach my $fileno (@filees) {
my $FH;
open $FH, '>', "/home/multifax/$fileno" or die "Can't open $fileno
!";
$handles{$fileno} = $FH;
}

...
if (grep $fields[4] eq $_, @filees) {
print {$handles{$_}} "$fields[1]\@$tmptxt\n";
}
...

foreach my $FH{values %handles} {
close $FH;
}


Jenda
=========== [email]Jenda@Krynicky.cz[/email] == [url]http://Jenda.Krynicky.cz[/url] ==========
There is a reason for living. There must be. I've seen it somewhere.
It's just that in the mess on my table ... and in my brain
I can't find it.
--- me

[Wolf Blaum]

For Quality purpouses, [email]LoneWolf@nc.rr.com[/email] 's mail on Wednesday 11 February
2004 22:08 may have been monitored or recorded as:

Hi,

> I'm still working on the script, though it is cleaning up even better, I
> even have it running cleaner by dumping anything not matching some specs.
> What I am trying to figure out though is this:
>
> I have 42 places for the output to go, 1 of those is a constant dump, the
> others are all based on whether or not there is data in a field. If the
> data is in the field, the data writes to a file with the same name as the
> data checked. If not then it writes to a global catch-all.

If I recall your last mail correctly you were opening a lot of file handles,
than running into the switch kind of thing and than closing all the files
again.
That was: a lot of system calls (open) to eventually write to a few of the
files in the SWITCH (the accumulated ifs) and then again a lot of sys calls
to closethem, where you have actually writen to only a few of the opend
files.

That sounds slow.
Wiggins allready suggested

if (grep $fields[4] == $_, @cities) {
* $tmptxt = $fields[10];
}
else {
* $tmptxt = '1-' . $fields[10];
}

for the first SWITCH like construct.
For the second one id say, make an array of your filenames and use the contend
of filed[11] as index, like:

my @file_names=qw (/home/multifax/everyone /home/multifax/
pack-fishbait .....);

if (defined $file_name[$fields[11] - 102]) {
open OUTFILE ">${file_name[$fields[11] - 102]}" or die "Cant open
${file_name[$fields[11] - 102]}:$!";

print OUTFILE "$fields[1]\@$tmptxt\n"; #your trailing ID
close OUTFILE;
}
else {
open OUTFILE ">default.out" or die "Cant open default.out :$!";
print OUTFILE "$fields[1]\@$tmptxt\n"; #your trailing ID
close OUTFILE;
}

However, this assumes that you have continous values from 102 upwards in
$fields[11] - if not, come up with a formula that gives you the index of the
wanted filename in @file_names depending on $fields[11]or use a hash instead:

$file_name{102}="whatever/filename/you.want";

I suggest you tell us, what the logic behind all these different files is, ie,
what goes where of which cause: maybe someone can come up with a hash
structure that incoorporates this knowledge.
Dont open and close 42 files if you only will ever print to 2 of them.

Enjoy, Wolf

[Shiping]

Hi,
How can I rearrange an array in a specific order based on the order of a
hash? Something like this:

my @a = qw(Mary John Dan);
print join "\t", @a, "\n";

my %b = ( John => 0,
Dan => 1,
Mary => 2);

print "$_ => $b{$_}\n" for (keys %b);
print "$_-$b{$_}\t" foreach sort {$b{$a} <=> $b{$b}} keys %b;

The final order for @a expect: John Dan Mary

Thanks,

Shiping

[John]

Shiping Wang wrote: 

Hello,
 


$ perl -le'
my @a = qw( Mary John Dan );
my %b = qw( John 0 Dan 1 Mary 2 );
print "@a";
@a = sort { $b{ $a } <=> $b{ $b } } @a;
print "@a";
'
Mary John Dan
John Dan Mary



John
--
use Perl;
program
fulfillment

[Jan]

John W. Krahn wrote:
 
>
>Hello,
> 
>
>
>$ perl -le'
>my @a = qw( Mary John Dan );
>my %b = qw( John 0 Dan 1 Mary 2 );
>print "@a";
>@a = sort { $b{ $a } <=> $b{ $b } } @a;
>print "@a";
>'
>Mary John Dan
>John Dan Mary
>[/ref]
Smart. But the sort pattern might be easier on the eye the array and hash are not named @a and %b.

$ perl -le'
my @array = qw( Mary John Dan );
my %hash = qw( John 0 Dan 1 Mary 2 );
print "@array";
@array = sort { $hash{ $a } <=> $hash{ $b } } @array;
print "@array";
'
Mary John Dan
John Dan Mary

This is mainly for my own better understanding.

- Jan
--
These are my principles and if you don't like them... well, I have others. - Groucho Marx

[Zentara]

On Mon, 16 Feb 2004 11:16:54 -0600, wustl.edu (Shiping
Wang) wrote:
 

I just asked a similar question on perlmonks, and was
given this gem, originally devised by broquiant.
################################################## #########
#!/usr/bin/perl
use warnings;
use strict;

my @unsorted = ( qw (Psdf lPik Easd aKwe SSdf eqwer scfgh Oegb rqwer T)
);
print join ',', @unsorted,"\n";

my %order = map {$_ => /[r-z]/ ? 'A' : /[a-q]/ ? 'B' : /[R-Z]/ ? 'C' :
'D'} ('a'..'z', 'A'..'Z');

my @sorted = map { substr($_ , 1) }
sort
map { $order{ substr($_,0,1) } . $_ } @unsorted;

print join ',', @sorted,"\n";
__END__


--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html