get list of files in subdirectories?

[Shaunn Johnson]

Howdy:

I'd like to get a list of files in a directory and
also in it's subdirectory - but I don't know how
far the subdirectory goes.

I can do a directory like so:

[snip]
#my $localdir=cwd;
my $localdir="/ddrive/db2_text/";
chdir $localdir;

opendir (DIR, $localdir) or die "can nae access D drive: $!";
local @ARGV = grep /\.txt$/, readdir DIR;
closedir (DIR);

[/snip]

But how can I get a list of files in a subdirectory without
having to specify all of the subdirectories as a variable?
(I hope that made sense).

It sounds like I'd have to do something like get a list of
subdirectories first and then do a search for .txt files
in each subdirectory?

Suggestions?

Thanks!

-X

[Rob Dixon]

Shaunn Johnson wrote:

>
> I'd like to get a list of files in a directory and
> also in it's subdirectory - but I don't know how
> far the subdirectory goes.
>
> I can do a directory like so:
>
> [snip]
> #my $localdir=cwd;
> my $localdir="/ddrive/db2_text/";
> chdir $localdir;
>
> opendir (DIR, $localdir) or die "can nae access D drive: $!";
> local @ARGV = grep /\.txt$/, readdir DIR;
> closedir (DIR);
>
> [/snip]
>
> But how can I get a list of files in a subdirectory without
> having to specify all of the subdirectories as a variable?
> (I hope that made sense).
>
> It sounds like I'd have to do something like get a list of
> subdirectories first and then do a search for .txt files
> in each subdirectory?

Hi Shaunn.

This is just the sort of job that File::Find was written for.
Try this short program:

use strict;
use warnings;

use File::Find;

my $localdir = '/ddrive/db2_text';

find(
sub { print $File::Find::name, "\n" if /\.txt$/ },
$localdir);

HTH,

Rob

[Drieux]

On Dec 23, 2003, at 11:52 AM, Johnson, Shaunn wrote:
[..]

> But how can I get a list of files in a subdirectory without
> having to specify all of the subdirectories as a variable?
> (I hope that made sense).
>
> It sounds like I'd have to do something like get a list of
> subdirectories first and then do a search for .txt files
> in each subdirectory?

[..]

Rob has offered up the classic File::Find solution.

Otherwise what you will want to do is build a tree
traverser yourself. You do not actually need to
chdir() into the directories - but would need
to remember that when you open the dir_block
that the 'short names' are not going to be the full path.

cf:
<http://www.wetware.com/drieux/pbl/Sys/FS/dir_walk.plx>

Personally I think going with the File::Find is
always the better long term solution - unless of
course one is trying to learn about how to deal with
things like recursion, and how to solve the classic
problem of how to check enroute to something else,
rather than having to solve

what are the sub_directories I will want to traverse
so that I can put them in an array...


ciao
drieux

---

[R. Joseph Newton]

drieux wrote:

> On Dec 23, 2003, at 11:52 AM, Johnson, Shaunn wrote:
> [..]

> > But how can I get a list of files in a subdirectory without
> > having to specify all of the subdirectories as a variable?
> > (I hope that made sense).
> >
> > It sounds like I'd have to do something like get a list of
> > subdirectories first and then do a search for .txt files
> > in each subdirectory?

> [..]
>
> Rob has offered up the classic File::Find solution.
>
> Otherwise what you will want to do is build a tree
> traverser yourself. You do not actually need to
> chdir() into the directories - but would need
> to remember that when you open the dir_block
> that the 'short names' are not going to be the full path.
>
> cf:
> <http://www.wetware.com/drieux/pbl/Sys/FS/dir_walk.plx>
>
> Personally I think going with the File::Find is
> always the better long term solution - unless of
> course one is trying to learn about how to deal with
> things like recursion, and how to solve the classic
> problem of how to check enroute to something else,
> rather than having to solve
>
> what are the sub_directories I will want to traverse
> so that I can put them in an array...
>
> ciao
> drieux

Thanks. I agree on both points. Given the tested and proven
nature of standard modules, it is definitely more responsible to
use them in production work. OTOH, the too-quick recourse to
packaged functionality can short-circuit the development of skills,
and more importantly, conceptual mastery, for the student. I see
recursion as an essential concept for developing programmers to
grasp and integrate, and a directory tree is probably the most
available practical example of its application. Whatever is
finally used in production code, the exercise of writing a
directory tree traversal is worthwhile in itself.

Joseph

[Drieux]

On Dec 24, 2003, at 1:23 PM, R. Joseph Newton wrote:

> drieux wrote:

>>
>> cf:
>> <http://www.wetware.com/drieux/pbl/Sys/FS/dir_walk.plx>

[..]

CAVEAT: this is not a call for the return to 'town/gown' kvetching.
the fact that I was victimized by latin, greek and hebrew
in my formative years is not meant to rally 'the old school ties'
against the heathens of the unwashed masses...

> OTOH, the too-quick recourse to packaged functionality can
> short-circuit the development of skills,
> and more importantly, conceptual mastery, for the student.

There is also the problem of

ceteris paribus ( latin for "all things being equal" )

that swings both ways. Many of the 'best practices' that
we have are derived from some set of personal experience,
or faith in the kvetching of another who is alledged to
have had a good reason for doing it that way.

If anyone drops that dir_walk.plx into their pet
perl debugger { which I highly recommend, nothing is
quite as Frightening a moment as watching what code
really does, than watching it roll over the rollers
in the perl debugger... besides building out one's debugging
skill set will impress your Friends, CoWorkers, and
GenderAppropriatePerkin }
they may pause and ask themselves about

DB<1>
File::Spec::Unix::canonpath(/System/Library/Perl/5.8.1/File/Spec/
Unix.pm:46):
46: if ( $^O =~ m/^(?:qnx|nto|cygwin)$/ && $path =~
s:^(//[^/]+)(/|\z):/:s ) {
DB<1>

and wonder why it is that if they were planning to only
use the code on a *nix box, they should have made the step
out to File::Spec on the off chance that it might some day
be used on a VMS box.

The problem here of course is that one has either stepped
off to 'isolate' their OS/FS specific code in a module so
that only that Module will need to be 'upgraded' to use
say File::Spec, and/or other OS specific Modules, Or one takes
the same ceteris paribus approach and opts to buy the
general case solution to begin with...

As a part of the ongoing struggle we note that the
code at
<http://www.wetware.com/drieux/pbl/Sys/FS/unix_dir_walk.plx>
differs only from the more generic code with:

< my $start_dir = "$ENV{HOME}/tmp/Wetware";
---

> my $start_dir = File::Spec->catdir( $ENV{HOME}, qw/tmp Wetware/ );

62c65
< my $path = "$dir/$filename" ;
---

> my $path = File::Spec->catfile( $dir, $filename );

So I am more than willing to support the canonical Jihaud Du Jure
about the orthodox model for coding...

{
subtle sub_text - both codes use
my @allfiles = File::Spec->no_upwards( readdir DIR );
}

MAJOR WARNING:

there is this minor issue with 'symbolic links' that can
become nasty - what I like to refer to as the 'jeff foxworthy
problem'[1][2] - since it is possible to have an entry of the form

looper -> ../../

that would create an infinite loop in a simple recursive
code as originally advertised. This will not be cleaned up
by simply using File::Spec - and folks should plan to
deal with this class of issues.

> I see recursion as an essential concept for developing programmers
> to grasp and integrate,

[..]

I had not intended to get into the
'jeff foxworthy problem' - since one of the
hardest functional part of doing 'recursion' is
making sure that all of the 'exit conditions' are
cleanly covered....

But since we were taking the time to step into the
problem of Production v. Pedagogy I thought I would
take the time to both raise the issue of 'when is it
optimization' and that way of course logically leads
me to the inevitable problem of getting out of logical loops.

So while it is easier to solve that in the simpler case
of file system looping, it is less clear how one exits
gracefully from the problem of 'ceteris paribus',
all things being equal, except that as one grows out
their experience they figure out which are their personal
pet collection of 'inclinations' and sticks to them to
avoid the problems they know about...


ciao
drieux

---

[1] for our non american reader's the gag is

you might be a redneck if your
family tree has any loops in it

tree_traversing needs to be able to manage this
sort of 'looping' issue - since whether it is a
file system that allows 'symbolic links' - or any
other form of hierarchical structure, if the system
allows for a 'loop'.... well, one needs to resolve
when it is time to get out of the loop...

[2] in my next life I get to get out of the habit of footnoting...