Counting files

[Shawn Corey]

Hi,

I have a list:

file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
....

I want to count them like:

$File{.}{file1} = count;
$File{.}{file2} = count;
$File{dir1}{file3} = count;
$File{dir1}{file4} = count;
$File{dir1}{subdir1}{file5} = count;
$file{dir2}{file6} = count;
....

Is the a nifty way of doing this in Perl-speak?

[Purl Gurl]

Shawn Corey wrote:

> I have a list:

> file1
> file2
> dir1/file3
> dir1/file4
> dir1/subdir1/file5
> dir2/file6

> I want to count them like:

> $File{.}{file1} = count;
> $File{.}{file2} = count;
> $File{dir1}{file3} = count;
> $File{dir1}{file4} = count;
> $File{dir1}{subdir1}{file5} = count;
> $file{dir2}{file6} = count;

Your article is gibberish.

Precisely what do you want to count?

Work towards developing an ability to write articles
which are clear, concise, coherent and free of gibberish.


Purl Gurl

[James Willmore]

On Tue, 09 Sep 2003 13:45:19 -0400
Shawn Corey <shawn@magma.ca> wrote:

> I have a list:
>
> file1
> file2
> dir1/file3
> dir1/file4
> dir1/subdir1/file5
> dir2/file6
> ...
>
> I want to count them like:
>
> $File{.}{file1} = count;
> $File{.}{file2} = count;
> $File{dir1}{file3} = count;
> $File{dir1}{file4} = count;
> $File{dir1}{subdir1}{file5} = count;
> $file{dir2}{file6} = count;
> ...
>
> Is the a nifty way of doing this in Perl-speak?

If you're just looking to count the entries in the list, you may want
to do ...

==untested==
my @list =
qw(file1 file2 dir1/file3 dir1/file4 dir1/subdir1/file5 dir2/file6 );
my $count = @list;
print "The number of file entries is: $count\n";

Is this what you were after?

HTH

--
Jim

Copyright notice: all code written by the author in this post is
released under the GPL. [url]http://www.gnu.org/licenses/gpl.txt[/url]
for more information.

a fortune quote ...
Trying to be happy is like trying to build a machine for which
the only specification is that it should run noiselessly.

[Greg Bacon]

In article <b9SdncIafYkfjMOiU-KYuQ@magma.ca>,
Shawn Corey <shawn@magma.ca> wrote:

: I have a list:
:
: file1
: file2
: dir1/file3
: dir1/file4
: dir1/subdir1/file5
: dir2/file6
: ...
:
: I want to count them like:
:
: $File{.}{file1} = count;
: $File{.}{file2} = count;
: $File{dir1}{file3} = count;
: $File{dir1}{file4} = count;
: $File{dir1}{subdir1}{file5} = count;
: $file{dir2}{file6} = count;
: ...
:
: Is the a nifty way of doing this in Perl-speak?

I don't know about nifty, but see below.

#! /usr/local/bin/perl

use strict;
use warnings;

use Data::Dumper;
use File::Find;

sub store {
my $file = shift;
my $count = shift;

return unless $file && @_;

my $hd = shift;
if (@_) {
$file->{$hd} ||= {};
store($file->{$hd}, $count, @_);
}
else {
$file->{$hd} = $$count++;
}
}

sub make_counter {
my $count = 0;
my %file;

my $sub = sub {
return unless -f;

store \%file, \$count, grep /./s,
split /\//, $File::Find::name;
};

(sub { %file }, $sub);
}

## main
die "Usage: $0 dir..\n" unless @ARGV;

my($file,$wanted) = make_counter;

find $wanted, @ARGV;

print Dumper $file->();

Hope this helps,
Greg
--
When in doubt, use brute force.
-- mjd, "Good Advice and Maxims for Programmers"

[Shawn Corey]

Purl Gurl wrote:

> Your article is gibberish.
>
> Precisely what do you want to count?
>
> Work towards developing an ability to write articles
> which are clear, concise, coherent and free of gibberish.
>
>
> Purl Gurl

OK, is this better?

1. I have a list of files like:
file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
....
They are stored in an array, call it @listOfFiles:
@listOfFiles = qw(
file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
....
);

2. The files are already stored in an array. I do not need to call
File::Find to get them. They are in @listOfFiles.

3. The files are already stored in an array. I do not need to call
opendir...; readdir...; closedir...; They are in @listOfFiles.

4. All the files are relative paths. None start with a slash.

5. All the files follow the UNIX convention. None have their directories
separated by a backslash. None of them follow the MS-DOS method of
separating directories in a path. If a backslash appears in a file, it
is part of the name.

6. For each file in the list of files, I want to build a hash of hashes
of hashes, one level for each directory in the path, with the last level
being the basename of the file, that count the number of times the file
appears in the list. The exception is that if there is no directories,
assume it has one (and only one) called '.'; if the file is "file1"
treat it as "./file1"

7. In other words, if the file is "file1" then execute a command that
has the same effect as: $File{ '.' }{ 'file1' } ++;

8. If the file has exactly one directory, called 'dir1', execute a
command that has the same effect as: $File{ 'dir1' }{ 'file3' } ++;

9. If the file has exactly two directories, called 'dir1' and 'subdir1',
execute a command that has the same effect as:
$File{ 'dir1' }{ 'subdir1' }{ 'file5' } ++;

10. Etc.

11. The number of directories may be any number and is not known until
the program is run. The number of directories is undeterminate but finite.

12. Question: is there some Perl command, like map, or some Perl module
I can use in the following algorithm?

foreach my $file ( @listOfFiles )
{
my @directory_or_basename = split /\//, $file;

# add the dot directory if there are none.
unshift @directory_or_basename, '.'
if @directory_or_basename == 1;

my $refHash = \%File;

while( my $directory_or_basename = shift @directory_or_basename )
{
if( @directory_or_basename ) # if this list is not empty,
# $directory_or_basename must
# be a directory.
{
$refHash->{ $directory_or_basename } = {}
unless defined $refHash->{ $directory_or_basename };

$refHash = $refHash->{ $directory_or_basename };
}
else # the list is empty,
# therefore $directory_or_basename must be a basename.
{
$refHash->{ $directory_or_basename } ++;
}
}
}

[Tom]

Shawn Corey <shawn@magma.ca> wrote in message news:<b9SdncIafYkfjMOiU-KYuQ@magma.ca>...

> Hi,
>
> I have a list:
>
> file1
> file2
> dir1/file3
> dir1/file4
> dir1/subdir1/file5
> dir2/file6
> ...
>
> I want to count them like:
>
> $File{.}{file1} = count;
> $File{.}{file2} = count;
> $File{dir1}{file3} = count;
> $File{dir1}{file4} = count;
> $File{dir1}{subdir1}{file5} = count;
> $file{dir2}{file6} = count;
> ...
>
> Is the a nifty way of doing this in Perl-speak?

If your list is from a directory, you can do this to count the number
of files in each directory:

#!/usr/bin/perl -w

use strict;
use File::stat;

my %counts;
countFiles($ARGV[0]);
foreach my $file(sort keys %counts)
{
print "$file = $counts{$file}\n";
}

sub countFiles
{
my ($directory) = $_[0];
my $fh;
if(! opendir $fh,$directory)
{
print "Error in accessing $directory.";
return;
}
while( defined(my $file=readdir($fh)))
{
next if $file =~ /^\./;
my $stat = stat("$directory/$file");
if($stat->mode()&040000) { countFiles("$directory/$file") }
else { $counts{"$directory"}++ }
}
closedir($fh);
}

The output generated will be like this:

/list = 2
/list/dir1 = 2
/list/dir1/subdir1 = 1
/list/dir2 = 1

If the output is in a format as you specified, there is no counting.

Tom
ztml.com

[Chief Squawtendrawpet]

Shawn Corey wrote:

> Is there some Perl command, like map, or some Perl module
> I can use in the following algorithm?

I don't have a ready-made solution for you or any brilliant
suggestions, but if you stick with your current algorithm you could
make two changes.

Your while() conditional won't do what you want if the dir evaluates
to false -- say, a directory named '0'.

Also, if speed is an issue, you could eliminate one of the
conditionals inside the while() loop like this:

$r = \%file;
while (@dir > 1){
$d = shift @dir;
$r->{$d} = {} unless defined $r->{$d};
$r = $r->{$d};
}
$r->{shift @dir} ++;

Chief S.

[John W. Krahn]

Shawn Corey wrote:

>
> 1. I have a list of files like:
> file1
> file2
> dir1/file3
> dir1/file4
> dir1/subdir1/file5
> dir2/file6
> ...
> They are stored in an array, call it @listOfFiles:
> @listOfFiles = qw(
> file1
> file2
> dir1/file3
> dir1/file4
> dir1/subdir1/file5
> dir2/file6
> ...
> );
>
> [snip]
>
> 4. All the files are relative paths. None start with a slash.
>
> 5. All the files follow the UNIX convention. None have their directories
> separated by a backslash. None of them follow the MS-DOS method of
> separating directories in a path. If a backslash appears in a file, it
> is part of the name.
>
> 6. For each file in the list of files, I want to build a hash of hashes
> of hashes, one level for each directory in the path, with the last level
> being the basename of the file, that count the number of times the file
> appears in the list. The exception is that if there is no directories,
> assume it has one (and only one) called '.'; if the file is "file1"
> treat it as "./file1"
>
> 7. In other words, if the file is "file1" then execute a command that
> has the same effect as: $File{ '.' }{ 'file1' } ++;
>
> 8. If the file has exactly one directory, called 'dir1', execute a
> command that has the same effect as: $File{ 'dir1' }{ 'file3' } ++;
>
> 9. If the file has exactly two directories, called 'dir1' and 'subdir1',
> execute a command that has the same effect as:
> $File{ 'dir1' }{ 'subdir1' }{ 'file5' } ++;

Based on your description, each path/filename will have a count of one.
If you want to base the count on the file name alone then you should the
file name as the key for the count.



John
--
use Perl;
program
fulfillment

[Shawn Corey]

John W. Krahn wrote:

> Based on your description, each path/filename will have a count of one.
> If you want to base the count on the file name alone then you should the
> file name as the key for the count.
>
>
>
> John

The path/filename should have a count of one, unless I screwed up
somewhere else in my program. One of the reason I was counting them was
a sanity check to ensure the other parts are working correctly.

[Shawn Corey]

Chief Squawtendrawpet wrote:

> I don't have a ready-made solution for you or any brilliant
> suggestions, but if you stick with your current algorithm you could
> make two changes.
>
> Your while() conditional won't do what you want if the dir evaluates
> to false -- say, a directory named '0'.

The only stupider than naming a directory '0' is to name one '*'.
*WARNING: DO NOT ATTEMPT THIS AT HOME. DO NOT ATTEMPT THIS AT WORK. DO
NOT ATTEMPT THIS ON ANY MACHINE YOU EVER TOUCH.*

>
> Also, if speed is an issue, you could eliminate one of the
> conditionals inside the while() loop like this:
>
> $r = \%file;
> while (@dir > 1){
> $d = shift @dir;
> $r->{$d} = {} unless defined $r->{$d};
> $r = $r->{$d};
> }
> $r->{shift @dir} ++;
>
> Chief S.

Thanks, that's the type of thing I am looking for.

[Mike Flannigan]

Tom wrote:

> If your list is from a directory, you can do this to count the number
> of files in each directory:
>
> #!/usr/bin/perl -w
>
> use strict;
> use File::stat;
>
> my %counts;
> countFiles($ARGV[0]);
> foreach my $file(sort keys %counts)
> {
> print "$file = $counts{$file}\n";
> }
>
> sub countFiles
> {
> my ($directory) = $_[0];
> my $fh;
> if(! opendir $fh,$directory)
> {
> print "Error in accessing $directory.";
> return;
> }
> while( defined(my $file=readdir($fh)))
> {
> next if $file =~ /^\./;
> my $stat = stat("$directory/$file");
> if($stat->mode()&040000) { countFiles("$directory/$file") }
> else { $counts{"$directory"}++ }
> }
> closedir($fh);
> }
>

That is pretty cool. I'm looking for something like this that will
simply output a list of all files to a text file, maybe along with
some size, date, etc. data. I could probably do this myself
with this script as a start. Thanks alot.


Mike

[Tom]

Shawn Corey <shawn@magma.ca> wrote in message news:<36adnSN34O_xj8KiU-KYgg@magma.ca>...
..
..

> > Your while() conditional won't do what you want if the dir evaluates
> > to false -- say, a directory named '0'.

> The only stupider than naming a directory '0' is to name one '*'.
> *WARNING: DO NOT ATTEMPT THIS AT HOME. DO NOT ATTEMPT THIS AT WORK. DO
> NOT ATTEMPT THIS ON ANY MACHINE YOU EVER TOUCH.*
>

..
..

Instead of warning someone not to do that, you should have just simply
taken care of this condition in your program yourself. If you tell
people not to do something, I'm sure someone will do it just to see
what would happen. BTW, I don't think anyone will be smart enough to
name the directory ‘*'.


#!/usr/bin/perl -w

use strict;

my @listOfFiles = qw(
file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
0/sub0/file7
);

my %File;
foreach my $file ( @listOfFiles )
{
my @dir = split /\//, $file;
unshift @dir, '.' if @dir == 1;
my $r = \%File;
while (@dir > 1)
{
my $d = shift @dir;
if(!$d)
{
print "WARNING: directory named '0' is renamed to NULL\n";
$d = "NULL";
}
$r->{$d} = {} unless defined $r->{$d};
$r = $r->{$d};
}
$r->{shift @dir} ++;
}
print "\n\nCOUNTS:\n\n";
opcounts("",\%File);
exit;

sub opcounts
{
my ($name,$files) = @_;

foreach my $file(sort keys %{$files})
{

if($name) { print "{$name}" } # print directory name

# is there a better way than the one below (checking for hash) ???
if($files->{$file} =~ /HASH/) { opcounts($file,$files->{$file}) }
else { print "{$file} = $files->{$file}\n" } # print file name and
total
}
}


Tom
ztml.com

[Tad McClellan]

Tom <tom@ztml.com> wrote:

> I don't think anyone will be smart enough to
> name the directory ‘*'.

^^^
^^^ supposed to be ?*


You don't have to be very smart at all:

perl -e "mkdir('?*')"


--
Tad McClellan SGML consulting
[email]tadmc@augustmail.com[/email] Perl programming
Fort Worth, Texas