pass cmd line file glob to grep for readdir

[qanda]

Hi all

I want to process lots of files (too many for the shell to expand on
one line) so I use readdir instead of glob. If I know the file glob
to use such as file*.ext I can put this directly into the grep ...
local @ARGV = grep /file.*\.ext/, readir CURDIR;
My question is how can I take a file glob on the command line and
transform it into an expression for grep. I suppose this also implies
we must prevent the shell from expanding by quoting the expression
such as myscript *file*.ext"

Thanks.

[Sam Holden]

On 8 Sep 2003 22:57:15 -0700, qanda <fumail@freeuk.com> wrote:

>
> My question is how can I take a file glob on the command line and
> transform it into an expression for grep. I suppose this also implies
> we must prevent the shell from expanding by quoting the expression
> such as myscript *file*.ext"

Why not use perl's glob function (perldoc -f glob)?

--
Sam Holden

[Eric J. Roode]

[email]fumail@freeuk.com[/email] (qanda) wrote in news:62b4710f.0309082157.4c16ee32
@posting.google.com:

> Hi all
>
> I want to process lots of files (too many for the shell to expand on

Have you considered using the xargs shell command?

--
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

[qanda]

> Why not use perl's glob function (perldoc -f glob)?

Because it won't work, as perldoc says, it returns the filename
expansions from the shell, the underlying problem is because the shell
has a limit in the number of characters allowed in one line (it may be
big, maybe 1MB) but with thousands of files, each with 30 or 40
character filenames that soon runs out. I don't want the user to try
using find and/or xargs, I just want to call myscript file*.ext or
quoted as in myscript "file*.ext" if need be.

Thanks.

[Tad McClellan]

Sam Holden <sholden@flexal.cs.usyd.edu.au> wrote:

> On 8 Sep 2003 22:57:15 -0700, qanda <fumail@freeuk.com> wrote:

>>
>> My question is how can I take a file glob on the command line and
>> transform it into an expression for grep. I suppose this also implies
>> we must prevent the shell from expanding by quoting the expression
>> such as myscript *file*.ext"

>
> Why not use perl's glob function (perldoc -f glob)?

This is a good idea if you have a perl version >= 5.6.0.

(otherwise the glob will fail because of the shell's limitations)


--
Tad McClellan SGML consulting
[email]tadmc@augustmail.com[/email] Perl programming
Fort Worth, Texas

[Tad McClellan]

Eric J. Roode <REMOVEsdnCAPS@comcast.net> wrote:

> [email]fumail@freeuk.com[/email] (qanda) wrote in news:62b4710f.0309082157.4c16ee32
> @posting.google.com:
>

>> Hi all
>>
>> I want to process lots of files (too many for the shell to expand on

>
> Have you considered using the xargs shell command?

This is a good idea if you have perl version < 5.6.0.


--
Tad McClellan SGML consulting
[email]tadmc@augustmail.com[/email] Perl programming
Fort Worth, Texas

[Sam Holden]

On 9 Sep 2003 05:53:29 -0700, qanda <fumail@freeuk.com> wrote:

>> Why not use perl's glob function (perldoc -f glob)?

>
> Because it won't work, as perldoc says, it returns the filename
> expansions from the shell, the underlying problem is because the shell
> has a limit in the number of characters allowed in one line (it may be
> big, maybe 1MB) but with thousands of files, each with 30 or 40
> character filenames that soon runs out. I don't want the user to try
> using find and/or xargs, I just want to call myscript file*.ext or
> quoted as in myscript "file*.ext" if need be.

Maybe you should use a perl from this century, since the text:

Beginning with v5.6.0, this operator is implemented using the
standard "File::Glob" extension. See File::Glob for details.

in the documentation seems to disagree with your claim.

It might also be worthwhile finding a non-broken shell.

; cd /dev
; echo * * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
| wc
1 128984 783012
; perl -le 'print join " ", <* * * * * * * * * * * * * * * * * * * * * * *'\
' * * * * * * * * * * * * * * * * * * * * * * *'\
' * * * * * * * * * * * * * * * * * * * * * * *'\
' * * * * * * * * * * * * * * * * * * * * * * *>' | wc
0 128984 783012
; /bin/echo * * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
| wc
bash: /bin/echo: Argument list too long

The limit is with the operating system, not the shell, so why should perl's
glob have any more difficulty than the shell's?

Have you actually tried it out? My 10 second test above seems to disagree
with your "it won't work" statement.

--
Sam Holden

[Tad McClellan]

qanda <fumail@freeuk.com> wrote:

>> Why not use perl's glob function (perldoc -f glob)?

>
> Because it won't work, as perldoc says, it returns the filename
> expansions from the shell,

Only with old perls.

What perl version do you have?


--
Tad McClellan SGML consulting
[email]tadmc@augustmail.com[/email] Perl programming
Fort Worth, Texas

[qanda]

Thanks guys and girls, I wasn't aware it had been improved in the new
versions; I'm limited (by customer) to using 5.005.

As for the argument limit, yes the underlying issue is with the OS, as
I understand it the limit is set by ARG_MAX in limits.h.

Would anyone be able to answer the (original) question for me,
certainly if using a newer version of Perl glob would be fine; however
I can't do that for this particular problem and the only overhead I
want for the user is to possibly quote the file glob so we avoid find
and/or xargs, etc.

Thanks again.

[qanda]

Sorry for posting on top of my last reply, but I really do need help
with this one. I will be forever (well quite some time at least) in
your debt if you can help :)