Opening two files at once

[denis@teachlinux.com]

I have two text files I would like to open and read at the same time.
Whats the best way to do this?

I was think along these lines:

open (TESTFILE, <"somefile.txt")|| die "can not open file";
open (LOGFILE, <"my_other_file")|| die "can not open other file";
while (($line1 = <TESTFILE>) & ($line2 = <LOGFILE>))
{
# print the line from each file
print "$line1\n $line2\n";
}
close TESTFILE;
close LOGFILE;


I know the while statement is wrong,

Whats the best way, and most understandable way?
and is there a way to do this without loading a module?

Thanks

Denis

[Charles K. Clarkson]

[email]denis@teachlinux.com[/email] <enis@teachlinux.com> wrote:
:
: I have two text files I would like to open and read at the
: same time. Whats the best way to do this?
:
: I was think along these lines:
:
: open (TESTFILE, "<somefile.txt")|| die "can not open file";

It is helpful to add the perl error code in there:

open (TESTFILE, "<somefile.txt") ||
die qq|Cannot open "somefile.txt": $!";

: open (LOGFILE, "<my_other_file")|| die "can not open other file";

It would be helpful to know what you're going to do
with the files. Do they need to be synchronized? That is,
do you need to access the same line of each file together
or do you need to access lines of one file and different
lines of the other?

Are they small files? Can they be safely loaded into
memory or are they going to be large files? Do you ever
need to go back to an old line you have already processed?
Will there ever be a need for more than three files?
Finally, are you using perl 5.6.1 or above?

You're going to get some helpful answers, but more
information about what you intend to do with the files
can aid us in finding better solutions.


HTH,

Charles K. Clarkson
--
Head Bottle Washer,
Clarkson Energy Homes, Inc.
Mobile Home Specialists
254 968-8328

[Drieux]

On Dec 22, 2003, at 4:32 PM, [email]denis@teachlinux.com[/email] wrote:

> I have two text files I would like to open and read at the same time.
> Whats the best way to do this?

the only problem with your current form, barring
the faux paux of

while( ($line1 = <TESTFILE>) && ($line2 = <LOGFILE>))
{

}

is what to do with the case of one file being longer than
the other file.

So you might try the idea of

my ($line1, $line2 , $flag );
while ( 1 ) {
$line1 = <TESTFILE>; # read them each
$line2 = <LOGFILE>;
last unless ( $line1 || $line2 ); # if both empty exit loop
#
# otherwise test for which one just finished
#
unless( $line1 )
{
$flag = 1;
last;
}
unless( $line2 )
{
$flag = 2;
last
}
#
# now do the voodo on the two lines
#
chomp($line2);
chomp($line1);
print "$line2 : $line1\n";
}
# if one needs to know which ended first one
# can deal with that here...
if ( $flag )
{
print "more lines to read\n";
print "\tfrom TestFile \n" if ( $flag == 1 );
print "\tfrom logfile \n" if ( $flag == 2 );
} else {
print "both ended together\n";
}
# now one would be safe to close the files

HTH.

ciao
drieux

---

[denis@teachlinux.com]

On Mon, 22 Dec 2003, Charles K. Clarkson wrote:

> [email]denis@teachlinux.com[/email] <enis@teachlinux.com> wrote:
> :
> : I have two text files I would like to open and read at the
> : same time. Whats the best way to do this?
> :
> : I was think along these lines:
> :
> : open (TESTFILE, "<somefile.txt")|| die "can not open file";
>
> It is helpful to add the perl error code in there:
>
> open (TESTFILE, "<somefile.txt") ||
> die qq|Cannot open "somefile.txt": $!";
>
> : open (LOGFILE, "<my_other_file")|| die "can not open other file";
>
> It would be helpful to know what you're going to do
> with the files. Do they need to be synchronized? That is,
> do you need to access the same line of each file together
> or do you need to access lines of one file and different
> lines of the other?
>
> Are they small files? Can they be safely loaded into
> memory or are they going to be large files? Do you ever
> need to go back to an old line you have already processed?
> Will there ever be a need for more than three files?
> Finally, are you using perl 5.6.1 or above?
>
> You're going to get some helpful answers, but more
> information about what you intend to do with the files
> can aid us in finding better solutions.
>
>
> HTH,
>
> Charles K. Clarkson
>

Thanks Charles,

The files are smaller then 1000 lines, so dropping thme into a memory
array would work.
Dont see ever haveing to go back and re-read them, other then re-running
the script. Would have to access the same line in each file at the same
time.

Runing Perl 5.8 on most systems, there are some systems that I'm not sure,
but will assume that there running at least 5.6.1

Will need to ensure that each file has the same number of lines included.
Was thinking of doing this via the "wc" command in unix.

The thing that really stinks is not being able to load modules on the
remote systems.

I did hack some code out last night that looks like this:

#!/usr/bin/perl -w
my @test1;
my @test2;
my $x = 0;

open (TEST1, "</etc/passwd")|| die " cant open passwd\n";
open (TEST2, "</etc/group")|| die " cant open group\n";
@test1 = <TEST1>;
@test2 = <TEST2>;
for ($x = 0; $x != $#test1; $x++)
{
print ($test1[$x]);
print ($test2[$x]);
print "\n";
}

there's not any line count check.. figured I'd use something like this..

# make sure the files have the same number of lines
@line_count = (`wc -l *tcp* *udp*`);
($lc, @rem) = split(" ",$line_count[0]);
for (@line_count)
{
($lc1_temp, @rem) = split (" ");
}
if ($lc1_temp/$#line_count != $lc)
{
die "\n\n\n Files do not have the same number of lines!\n\n\n";
}

Thanks,

Denis

[denis@teachlinux.com]

On Mon, 22 Dec 2003, drieux wrote:

>
> On Dec 22, 2003, at 4:32 PM, [email]denis@teachlinux.com[/email] wrote:
>

> > I have two text files I would like to open and read at the same time.
> > Whats the best way to do this?

>
> the only problem with your current form, barring
> the faux paux of
>
> while( ($line1 = <TESTFILE>) && ($line2 = <LOGFILE>))
> {
>
> }
>
> is what to do with the case of one file being longer than
> the other file.
>
> So you might try the idea of
>
> my ($line1, $line2 , $flag );
> while ( 1 ) {
> $line1 = <TESTFILE>; # read them each
> $line2 = <LOGFILE>;
> last unless ( $line1 || $line2 ); # if both empty exit loop
> #
> # otherwise test for which one just finished
> #
> unless( $line1 )
> {
> $flag = 1;
> last;
> }
> unless( $line2 )
> {
> $flag = 2;
> last
> }
> #
> # now do the voodo on the two lines
> #
> chomp($line2);
> chomp($line1);
> print "$line2 : $line1\n";
> }
> # if one needs to know which ended first one
> # can deal with that here...
> if ( $flag )
> {
> print "more lines to read\n";
> print "\tfrom TestFile \n" if ( $flag == 1 );
> print "\tfrom logfile \n" if ( $flag == 2 );
> } else {
> print "both ended together\n";
> }
> # now one would be safe to close the files
>
> HTH.
>
> ciao
> drieux
>
> ---
>
>
>

Thanks, looks pretty good..

[denis@teachlinux.com]

Drieux,

Sorry for the curt thank you. You code looks really good.

On Mon, 22 Dec 2003, drieux wrote:

>
> On Dec 22, 2003, at 4:32 PM, [email]denis@teachlinux.com[/email] wrote:
>

> > I have two text files I would like to open and read at the same time.
> > Whats the best way to do this?

>
> the only problem with your current form, barring
> the faux paux of
>
> while( ($line1 = <TESTFILE>) && ($line2 = <LOGFILE>))
> {
>
> }
>
> is what to do with the case of one file being longer than
> the other file.
>
> So you might try the idea of
>
> my ($line1, $line2 , $flag );
> while ( 1 ) {
> $line1 = <TESTFILE>; # read them each
> $line2 = <LOGFILE>;

never thought of reading two files one line at a time like this, looks so
easy once you showed me.

> last unless ( $line1 || $line2 ); # if both empty exit loop
> #
> # otherwise test for which one just finished
> #
> unless( $line1 )
> {
> $flag = 1;
> last;
> }
> unless( $line2 )
> {
> $flag = 2;
> last
> }

a much better line check then the "junk" I wrote.

> #
> # now do the voodo on the two lines
> #
> chomp($line2);
> chomp($line1);
> print "$line2 : $line1\n";
> }
> # if one needs to know which ended first one
> # can deal with that here...
> if ( $flag )
> {
> print "more lines to read\n";
> print "\tfrom TestFile \n" if ( $flag == 1 );
> print "\tfrom logfile \n" if ( $flag == 2 );
> } else {
> print "both ended together\n";
> }
> # now one would be safe to close the files
>
> HTH.
>

Helps a bunch,

Thanks Again to all who helped..

Denis

> ciao
> drieux
>
> ---
>
>
>

[Drieux]

On Dec 23, 2003, at 9:03 AM, [email]denis@teachlinux.com[/email] wrote:
[..]

> Sorry for the curt thank you. You code looks really good.

I'd never thought of it that way. What I think might
be the elegant way of saying 'thank you' to folks who
offer an idea/code/suggestion/BriefPsychoticEpisode would
be something on the order of

a. demonstrating what one had 'learned' from the code, idea, ....
b. being able to help others with what one learned to others.

Often times folks don't have the time, due to the press
of what ever it was that got them tossing a 'current issue'
into play...

> On Mon, 22 Dec 2003, drieux wrote:

>> On Dec 22, 2003, at 4:32 PM, [email]denis@teachlinux.com[/email] wrote:

[..]

>>
>> So you might try the idea of
>>
>> my ($line1, $line2 , $flag );
>> while ( 1 ) {
>> $line1 = <TESTFILE>; # read them each
>> $line2 = <LOGFILE>;

>
> never thought of reading two files one line at a time like this,
> looks so easy once you showed me.

the gooder news is that it 'solves' the problem
you asked about - the information is that it is
still not clear which way you really want to go.

you might check out MJD's 'diff' in perl:
<http://www.perl.com/language/ppt/src/diff/diff.mjd>

>

>> last unless ( $line1 || $line2 ); # if both empty exit loop
>> #
>> # otherwise test for which one just finished
>> #
>> unless( $line1 )
>> {
>> $flag = 1;
>> last;
>> }
>> unless( $line2 )
>> {
>> $flag = 2;
>> last
>> }

> a much better line check then the "junk" I wrote.

actually upon reflection I thought

$flag = 1 unless( $line1 );
$flag = 2 unless( $line2 );
last if $flag;

would solve that part - but had it fanned out
in case you wanted to do something specific with
either case....

The trade offs are basically simple:

a. slurp in both files using a common slurper function
then call a compare_slurpie()

b. find a neater data model that will allow you to
parse the files for the information you need, write
one grot_data_from_file() method, call it twice and
then a compare_grotted_data()

c. pass in two file_names and grovel in parrallel as is
line by line...
[..]

ciao
drieux

---

[denis@teachlinux.com]

On Tue, 23 Dec 2003, drieux wrote:

>
> On Dec 23, 2003, at 9:03 AM, [email]denis@teachlinux.com[/email] wrote:
> [..]

> > Sorry for the curt thank you. You code looks really good.

>
> I'd never thought of it that way. What I think might
> be the elegant way of saying 'thank you' to folks who
> offer an idea/code/suggestion/BriefPsychoticEpisode would
> be something on the order of
>
> a. demonstrating what one had 'learned' from the code, idea, ....
> b. being able to help others with what one learned to others.
>
> Often times folks don't have the time, due to the press
> of what ever it was that got them tossing a 'current issue'
> into play...
>

> > On Mon, 22 Dec 2003, drieux wrote:

> >> On Dec 22, 2003, at 4:32 PM, [email]denis@teachlinux.com[/email] wrote:

> [..]

> >>
> >> So you might try the idea of
> >>
> >> my ($line1, $line2 , $flag );
> >> while ( 1 ) {
> >> $line1 = <TESTFILE>; # read them each
> >> $line2 = <LOGFILE>;

> >
> > never thought of reading two files one line at a time like this,
> > looks so easy once you showed me.

>
> the gooder news is that it 'solves' the problem
> you asked about - the information is that it is
> still not clear which way you really want to go.
>
> you might check out MJD's 'diff' in perl:
> <http://www.perl.com/language/ppt/src/diff/diff.mjd>
>

> >

> >> last unless ( $line1 || $line2 ); # if both empty exit loop
> >> #
> >> # otherwise test for which one just finished
> >> #
> >> unless( $line1 )
> >> {
> >> $flag = 1;
> >> last;
> >> }
> >> unless( $line2 )
> >> {
> >> $flag = 2;
> >> last
> >> }

> > a much better line check then the "junk" I wrote.

>
> actually upon reflection I thought
>
> $flag = 1 unless( $line1 );
> $flag = 2 unless( $line2 );
> last if $flag;
>
> would solve that part - but had it fanned out
> in case you wanted to do something specific with
> either case....
>
> The trade offs are basically simple:
>
> a. slurp in both files using a common slurper function
> then call a compare_slurpie()
>
> b. find a neater data model that will allow you to
> parse the files for the information you need, write
> one grot_data_from_file() method, call it twice and
> then a compare_grotted_data()
>
> c. pass in two file_names and grovel in parrallel as is
> line by line...
> [..]
>
> ciao
> drieux
>
> ---
>
>
>

Not really sure as to the final use of the code. I have a perl script I
wrote back in Sept that opens a file, reads the file and runs a list of
tests to do regression testing on LA-MPI (Los Alamos MPI). Currently I
have a "kill" timer that runs during each test and kills the running test
if it has a timeout. What I'm looking at is changing the timer depending
on the test. Sounds simple enough, but I dont want to have to go in and modify
the regression test list, it's almost 500 tests, and can change with
each run.

One of the issues is that I sometimes have to contend with other users on
the system, so tests time out, even if they are running fine.. Was
thinking along the lines of opening the tests file and also a stored time_to_run_tests file, double the value in
the time_to_run_tests, and us it as the timeout value. However there may
be times when there are not any time_to_run_tests files to use..

So I'm just figuring it out as I go, and asking dumb questions..

Thanks

Denis