Dir.foreach not with patterns?

[Kurt V. Hindenburg]

--Boundary-02=_GKgY/p64avOG7gO
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

Why does the third one not work as expected? It appears that=20
Dir.foreach will not work with patterns...

#!/usr/bin/ruby

s =3D "/etc/host*"

#1
l =3D Dir[s]=20
p l

#2
Dir.foreach("/etc") { |d| p d }

#3
Dir.foreach(s) { |d| p d }

#4
# This is wasteful
l =3D Dir[s]
l.each { |f| p f }

=2D-=20
^^^ Kurt

There is no good nor evil, just power.

--Boundary-02=_GKgY/p64avOG7gO
Content-Type: application/pgp-signature
Content-Description: signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQA/YgKG0cAvx3ELfKARAt8dAJ4wrTeNfjRnQZ9XrserhejBSuNGKA CggWAk
WAnLflpyjLYvLKOZfQH4fcQ=
=Fd1o
-----END PGP SIGNATURE-----

--Boundary-02=_GKgY/p64avOG7gO--

[nobu.nokada@softhome.net]

Hi,

At Sat, 13 Sep 2003 02:28:02 +0900,
Kurt V. Hindenburg wrote:

> Why does the third one not work as expected? It appears that
> Dir.foreach will not work with patterns...

Use Dir.glob.

--
Nobu Nakada

[dblack@superlink.net]

Hi --

On Sat, 13 Sep 2003, Kurt V. Hindenburg wrote:

> Why does the third one not work as expected? It appears that
> Dir.foreach will not work with patterns...
>
> #!/usr/bin/ruby
>
> s = "/etc/host*"
>
> #1
> l = Dir[s]
> p l
>
> #2
> Dir.foreach("/etc") { |d| p d }
>
> #3
> Dir.foreach(s) { |d| p d }
>
> #4
> # This is wasteful
> l = Dir[s]
> l.each { |f| p f }

You don't need two lines, though -- you can do this in a way that's actually
shorter than #3:

Dir[s].each {|f| p f}


David

--
David Alan Black
home: [email]dblack@superlink.net[/email]
work: [email]blackdav@shu.edu[/email]
Web: [url]http://pirate.shu.edu/~blackdav[/url]

[Weirich, James]

I like the Dir[] form (or its "glob" alternative). I used to write
recursive functions to search a directory tree. Then I discovered
Dir['**/*.rb']. Not only is it easier, it's also faster.

--
-- Jim Weirich / Compuware
-- FWP Capture Services
-- Phone: 859-386-8855

> -----Original Message-----
> From: [email]dblack@superlink.net[/email] [mailto:dblack@superlink.net]
> Sent: Friday, September 12, 2003 1:44 PM
> To: [email]ruby-talk@ruby-lang.org[/email]
> Subject: Re: Dir.foreach not with patterns?
>
>
> Hi --
>
> On Sat, 13 Sep 2003, Kurt V. Hindenburg wrote:
>

> > Why does the third one not work as expected? It appears that
> > Dir.foreach will not work with patterns...
> >
> > #!/usr/bin/ruby
> >
> > s = "/etc/host*"
> >
> > #1
> > l = Dir[s]
> > p l
> >
> > #2
> > Dir.foreach("/etc") { |d| p d }
> >
> > #3
> > Dir.foreach(s) { |d| p d }
> >
> > #4
> > # This is wasteful
> > l = Dir[s]
> > l.each { |f| p f }

>
> You don't need two lines, though -- you can do this in a way
> that's actually
> shorter than #3:
>
> Dir[s].each {|f| p f}
>
>
> David
>
> --
> David Alan Black
> home: [email]dblack@superlink.net[/email]
> work: [email]blackdav@shu.edu[/email]
> Web: [url]http://pirate.shu.edu/~blackdav[/url]
>
>

[Alan Davies]

Weirich, James wrote:

>> I like the Dir[] form (or its "glob" alternative). I used to write
>> recursive functions to search a directory tree. Then I discovered
>> Dir['**/*.rb']. Not only is it easier, it's also faster.

I'm not so sure. I timed my recursive "Filescan" class against Dir[] on
a large fileserver. Dir[] took 92 seconds, and my Filescan class only
took 72 seconds.

I also prefer the ordering of the output of my Filescan class.

[Weirich, James]

> >> I like the Dir[] form (or its "glob" alternative). I used to write

> >> recursive functions to search a directory tree. Then I discovered
> >> Dir['**/*.rb']. Not only is it easier, it's also faster.

>
> I'm not so sure. I timed my recursive "Filescan" class against Dir[] on
> a large fileserver. Dir[] took 92 seconds, and my Filescan class only
> took 72 seconds.

Hmmm ... Perhaps I implemented the my recursive search in a slow manner.
Care to share your Filescan class?

--
-- Jim Weirich / Compuware
-- FWP Capture Services
-- Phone: 859-386-8855

[Jason Creighton]

On Thu, 18 Sep 2003 15:59:25 +0100
Alan Davies <NOSPAMcs96and@yahoo.co.ukNOSPAM> wrote:

> Weirich, James wrote:
>

> >> I like the Dir[] form (or its "glob" alternative). I used to write
> >> recursive functions to search a directory tree. Then I discovered
> >> Dir['**/*.rb']. Not only is it easier, it's also faster.

>
> I'm not so sure. I timed my recursive "Filescan" class against Dir[] on
> a large fileserver. Dir[] took 92 seconds, and my Filescan class only
> took 72 seconds.
>
> I also prefer the ordering of the output of my Filescan class.

Could you post your Filescan class?

I'd like to see it, and compare w/ find.rb in the standard distribution.

Jason Creighton

[Alan Davies]

> Could you post your Filescan class?

>
> I'd like to see it, and compare w/ find.rb in the standard distribution.
>

Attatched. Its faster at most things than Dir#[]. I think the reason
it is faster is because it never actually changes the working directory.

To use it, e.g.:

require "Filescan"
File.open("out.txt", 'w') do |file|
Filescan.new("c:/").each { |name| file.puts name if name =~ /\.mp3$/i}
end

There are also functions each_filename, each_file, each_dirname, each_dir.

each_line can also be used to seach through indvidual lines of text
files. It can also be used for search and replace.


# Filescan.rb
#
# Class to recursively scan through files from a given dir.
#
# Copyright (C) 2003 by Alan Davies
#
# This software is provided 'as-is', without any express or implied
# warranty. In no event will the authors be held liable for any damages
# arising from the use of this software.
#
# Permission is granted to anyone to use this software for any purpose,
# including commercial applications, and to alter it and redistribute it
# freely, subject to the following restrictions:
#
# 1. The origin of this software must not be misrepresented; you must not
# claim that you wrote the original software. If you use this software
# in a product, an acknowledgment in the product documentation would be
# appreciated but is not required.
# 2. Altered source versions must be plainly marked as such, and must not be
# misrepresented as being the original software.
# 3. This notice may not be removed or altered from any source distribution.

class Filescan
include Enumerable

@@dotDir = /^\.\.?$/

def initialize(dir='.', recursive=true, verbose=false)
@startDir = dir
@recursive = recursive
@verbose = verbose

yield self if block_given?
end

attr_accessor :startDir, :recursive, :verbose

def each(startDir=@startDir, &block)
puts startDir if @verbose
begin
Dir.open(startDir) do |dir|
dir.each do |entry|
next if entry =~ @@dotDir
yield (fullPath = File.expand_path(entry, startDir))
self.each(fullPath, &block) if @recursive and File.directory?(fullPath)
end
end
rescue Errno::EINVAL, Errno::ENOENT
$stderr.puts "Can't open dir: #{startDir}"
return
end
end # def each

# Executes a block for each directory name
def each_dirname(pattern=//)
self.each { |dirName| yield dirName if File.directory?(dirName) and (dirName =~ pattern) }
end

# Executes a block for each directory
def each_dir(pattern=//)
each_dirname(pattern) do |dirName|
Dir.open(dirName) { |dirHandle| yield dirHandle, dirName }
end
end

# Execute a block for each filename
def each_filename(pattern=//)
self.each { |fileName| yield fileName if File.file?(fileName) and (fileName =~ pattern) }
end

# Execute a block for each file
def each_file(mode='r', pattern=//)
self.each_filename(pattern) do |filename|
File.open(filename, mode) { |file| yield file, filename }
end
end

# Executes a block for each line of each file.
# If the files are opened in writable mode, then the block must return the
# no of changes that were made to the line.
# Files are not written back to the disk if no lines are altered.
def each_line(writable=false, pattern=//)
self.each_file(writable ? 'r+' : 'r', pattern) do |file, filename|
fileChanges = lineChanges = len = 0
text = file.readlines

# Yield the block for each line in the file
text.each do |line|
lineChanges = yield(line, filename)
fileChanges += lineChanges if writable
end

# Write the file back to disk
if writable and (fileChanges > 0)
puts "#{filename} - #{fileChanges} changes made"
file.rewind
file.syswrite(text)
file.truncate(file.pos)
end
end # self.eachfile
end # def each_line

end # class Filescan

[Alan Davies]

> Attatched. Its faster at most things than Dir#[]. I think the reason

> it is faster is because it never actually changes the working directory.

And again. This one has much faster implementations of each_filename
and each_dirname, by doing the regexp before doing the file/directory test.

# Filescan.rb
#
# Class to recursively scan through files from a given dir.
#
# Copyright (C) 2003 by Alan Davies
#
# This software is provided 'as-is', without any express or implied
# warranty. In no event will the authors be held liable for any damages
# arising from the use of this software.
#
# Permission is granted to anyone to use this software for any purpose,
# including commercial applications, and to alter it and redistribute it
# freely, subject to the following restrictions:
#
# 1. The origin of this software must not be misrepresented; you must not
# claim that you wrote the original software. If you use this software
# in a product, an acknowledgment in the product documentation would be
# appreciated but is not required.
# 2. Altered source versions must be plainly marked as such, and must not be
# misrepresented as being the original software.
# 3. This notice may not be removed or altered from any source distribution.

class Filescan
include Enumerable

@@dotDir = /^\.\.?$/

def initialize(dir='.', recursive=true, verbose=false)
@startDir = dir
@recursive = recursive
@verbose = verbose

yield self if block_given?
end

attr_accessor :startDir, :recursive, :verbose

def each(startDir=@startDir, &block)
puts startDir if @verbose
begin
Dir.open(startDir) do |dir|
dir.each do |entry|
next if entry =~ @@dotDir
yield (fullPath = File.expand_path(entry, startDir))
self.each(fullPath, &block) if @recursive and File.directory?(fullPath)
end
end
rescue Errno::EINVAL, Errno::ENOENT
$stderr.puts "Can't open dir: #{startDir}"
return
end
end # def each

# Executes a block for each directory name
def each_dirname(pattern=//)
self.each { |dirName| yield dirName if (dirName =~ pattern) and File.directory?(dirName) }
end

# Executes a block for each directory
def each_dir(pattern=//)
each_dirname(pattern) do |dirName|
Dir.open(dirName) { |dirHandle| yield dirHandle, dirName }
end
end

# Execute a block for each filename
def each_filename(pattern=//)
self.each { |fileName| yield fileName if (fileName =~ pattern) and File.file?(fileName) }
end

# Execute a block for each file
def each_file(mode='r', pattern=//)
self.each_filename(pattern) do |filename|
File.open(filename, mode) { |file| yield file, filename }
end
end

# Executes a block for each line of each file.
# If the files are opened in writable mode, then the block must return the
# no of changes that were made to the line.
# Files are not written back to the disk if no lines are altered.
def each_line(writable=false, pattern=//)
self.each_file(writable ? 'r+' : 'r', pattern) do |file, filename|
fileChanges = lineChanges = len = 0
text = file.readlines

# Yield the block for each line in the file
text.each do |line|
lineChanges = yield(line, filename)
fileChanges += lineChanges if writable
end

# Write the file back to disk
if writable and (fileChanges > 0)
puts "#{filename} - #{fileChanges} changes made"
file.rewind
file.syswrite(text)
file.truncate(file.pos)
end
end # self.eachfile
end # def each_line

end # class Filescan

[ts]

>>>>> "A" == Alan Davies <NOSPAMcs96and@yahoo.co.ukNOSPAM> writes:


Change this

A> @@dotDir = /^\.\.?$/

You can have a file (or a directory) with \n in the name



Guy Decoux

[Alan Davies]

ts wrote:

>>>>>>"A" == Alan Davies <NOSPAMcs96and@yahoo.co.ukNOSPAM> writes:

>
>
>
> Change this
>
> A> @@dotDir = /^\.\.?$/
>
> You can have a file (or a directory) with \n in the name
>
>
>
> Guy Decoux
>

I'm not sure what you're getting at.

I'm pretty sure you can't have filenames with \n in them, and that
pattern wouldn't match them anyway.

[ts]

>>>>> "A" == Alan Davies <NOSPAMcs96and@yahoo.co.ukNOSPAM> writes:

A> I'm pretty sure you can't have filenames with \n in them, and that
A> pattern wouldn't match them anyway.

Then test it with a real OS :-)

svg% ls -a
./ ../
svg%

svg% ruby -e 'Dir.mkdir "aa\n.."'
svg%

svg% ls -a
./ ../ aa?../
svg%

svg% ruby -e 'Dir.foreach(".") {|d| p d}'
"."
".."
"aa\n.."
svg%

svg% ruby -e 'Dir.foreach(".") {|d| next if d =~ /^\.\.?$/; p d}'
svg%



Guy Decoux

[Alan Davies]

ts wrote:

>>>>>>"A" == Alan Davies <NOSPAMcs96and@yahoo.co.ukNOSPAM> writes:

>
>
> A> I'm pretty sure you can't have filenames with \n in them, and that
> A> pattern wouldn't match them anyway.
>
> Then test it with a real OS :-)

Personally I can't stand linux/unix, but each to his own, as they say.

>
> svg% ls -a
> / ../
> svg%
>
> svg% ruby -e 'Dir.mkdir "aa\n.."'
> svg%
>
> svg% ls -a
> / ../ aa?../
> svg%
>
> svg% ruby -e 'Dir.foreach(".") {|d| p d}'
> "."
> ".."
> "aa\n.."
> svg%
>
> svg% ruby -e 'Dir.foreach(".") {|d| next if d =~ /^\.\.?$/; p d}'
> svg%
>
>
>
> Guy Decoux
>
>

Filenames with \n in them is the most ridiculous thing I've ever heard.

I take it the regexp should be /\A\.\.?\z/ then?

Alan.

[ts]

>>>>> "A" == Alan Davies <NOSPAMcs96and@yahoo.co.ukNOSPAM> writes:

A> Filenames with \n in them is the most ridiculous thing I've ever heard.

:-)

A> I take it the regexp should be /\A\.\.?\z/ then?

or you can do like in lib/find.rb

svg% grep 'next if' lib/find.rb
next if f == "." or f == ".."
svg%




Guy Decoux

[Alan Davies]

ts wrote:

> next if f == "." or f == ".."

I did used to do that. I can't even remember why I changed it to a regexp!

[Gavin Sinclair]

On Wednesday, September 24, 2003, 12:33:54 AM, Alan wrote:

> ts wrote:

>> next if f == "." or f == ".."

> I did used to do that. I can't even remember why I changed it to a regexp!

Come on, you can't resist.

class Object
def in?(enum)
enum.include?(self)
end
end

...
next if f.in? [".", ".."]
...

Isn't that much nicer? ;)

Gavin

[Paul Duncan]

--TN8pJM9vJMHHFgJc
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* Gavin Sinclair (gsinclair@soyabean.com.au) wrote:

> On Wednesday, September 24, 2003, 12:33:54 AM, Alan wrote:
>=20

> > ts wrote:

> >> next if f =3D=3D "." or f =3D=3D ".."

>=20

> > I did used to do that. I can't even remember why I changed it to a reg=

exp!

>=20
> Come on, you can't resist.
>=20
> class Object
> def in?(enum)
> enum.include?(self)
> end
> end
>=20
> ...
> next if f.in? [".", ".."]
> ...

or
next if %w{. ..}.include? f

but personally i prefer this:

next if f =3D~ /^\.\.?$/

> Isn't that much nicer? ;)
> =20
> Gavin
>=20

--=20
Paul Duncan <pabs@pablotron.org> pabs in #gah (OPN IRC)
[url]http://www.pablotron.org/[/url] OpenPGP Key ID: 0x82C29562

--TN8pJM9vJMHHFgJc
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/eHGkzdlT34LClWIRAnFHAKCxLkNaqu4WcOdYd64j8v2FdX/CfgCgxVgP
BPrO3S7ZwUsVTK1nLL1EUpM=
=Isbc
-----END PGP SIGNATURE-----

--TN8pJM9vJMHHFgJc--

[dblack@superlink.net]

Hi --

On Tue, 30 Sep 2003, Paul Duncan wrote:

> * Gavin Sinclair (gsinclair@soyabean.com.au) wrote:

> > On Wednesday, September 24, 2003, 12:33:54 AM, Alan wrote:
> >

> > > ts wrote:
> > >> next if f == "." or f == ".."

> >

> > > I did used to do that. I can't even remember why I changed it to a regexp!

> >
> > Come on, you can't resist.
> >
> > class Object
> > def in?(enum)
> > enum.include?(self)
> > end
> > end
> >
> > ...
> > next if f.in? [".", ".."]
> > ...

>
> or
> next if %w{. ..}.include? f
>
> but personally i prefer this:
>
> next if f =~ /^\.\.?$/

However....

$ ruby -e 'File.new("abc\n..\ndef","w")'
$ ruby -e 'Dir["abc*"].each {|f|

> puts "#{f}: yes" if /^\.\.?$/.match(f)}'

abc
..
def: yes


I know, I know, not likely :-) Still, /\A\.\.?\z/ would be safer.


David

--
David Alan Black
home: [email]dblack@superlink.net[/email]
work: [email]blackdav@shu.edu[/email]
Web: [url]http://pirate.shu.edu/~blackdav[/url]