using a filter inside Ruby

[Eric Schwartz]

I've the contents of a raw log file in memory, and a program that will
parse this raw log data and print out human-readable info. The
problem is, I can't figure out how to use this program as a filter in
Ruby.

I've tried:

IO.popen('/usr/games/jive', 'w+') { |io|
io.puts "What is going on?"
puts io.gets
}

But it just hangs at the gets. This is nagging at the back of my
mind, and if I had my copy of _The Unix Programming Environment_, I
bet I'd find it in there, but it's on loan at the moment.

I could, if no other solution presents itself, write the raw log to a
file, run the filter on the file, and then read that data back into
ruby, but this strikes me as inefficient and error-prone. Can it be
done the way I'm trying to?

-=Eric
--
Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
-- Blair Houghton.

[Eric Schwartz]

Eric Schwartz <emschwar@pobox.com> writes:

> I've tried:
>
> IO.popen('/usr/games/jive', 'w+') { |io|
> io.puts "What is going on?"
> puts io.gets
> }
>
> But it just hangs at the gets.

Okay, after a fair amount of searching, including some blind guessing,
I came across some old ruby-talk posts mentioning Open3, which seems
to do what I want:

Open3.popen3('jive') { |wtr,rdr,err|
wtr.puts "what is up?"
wtr.close
puts rdr.gets
}

This points up my major frustration with Ruby-- I had to completely
guess as to how Open3 worked, or *even that it existed at all*. There
are no docs I can find for it, and this makes developing with Ruby
more of a pain than it needs to be. If I were doing this in perl, I'd
just have to 'perldoc -q pipe', and instantly I have docs on
IPC::Open2 and a reference to IPC::Open3 if I need that.

Pardon the gripiness, please, but I really hate guessing about my
programming language. POLS is nice, but having easily accessible docs
to confirm all the corner cases is even better, IMO.

-=Eric
--
Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
-- Blair Houghton.

[Mark Wilson]

On Tuesday, September 9, 2003, at 04:26 PM, Eric Schwartz wrote:

> [snip]

> This points up my major frustration with Ruby-- I had to completely
> guess as to how Open3 worked, or *even that it existed at all*. There
> are no docs I can find for it, and this makes developing with Ruby
> more of a pain than it needs to be. [snip]

I agree. Continued efforts documenting Ruby (in multiple languages) are
required.

Open3 is 'rdoc-able', so I'm sure this will get done (at least in
English).

I also think that IO and pipe questions are the least well documented
parts of Ruby.

Regards,

Mark

[Jason Creighton]

On Tue, 09 Sep 2003 14:11:19 -0600
Eric Schwartz <emschwar@pobox.com> wrote:

> Eric Schwartz <emschwar@pobox.com> writes:

> > I've tried:
> >
> > IO.popen('/usr/games/jive', 'w+') { |io|
> > io.puts "What is going on?"
> > puts io.gets
> > }
> >
> > But it just hangs at the gets.

>
> Okay, after a fair amount of searching, including some blind guessing,
> I came across some old ruby-talk posts mentioning Open3, which seems
> to do what I want:
>
> Open3.popen3('jive') { |wtr,rdr,err|
> wtr.puts "what is up?"
> wtr.close
> puts rdr.gets
> }
>
> This points up my major frustration with Ruby-- I had to completely
> guess as to how Open3 worked, or *even that it existed at all*. There
> are no docs I can find for it, and this makes developing with Ruby
> more of a pain than it needs to be. If I were doing this in perl, I'd
> just have to 'perldoc -q pipe', and instantly I have docs on
> IPC::Open2 and a reference to IPC::Open3 if I need that.

Yes, you're right, the documentation should be better. Of course, if you
look at open3.rb, you'll see this:

# Usage:
# require "open3"
#
# in, out, err = Open3.popen3('nroff -man')
# or
# include Open3
# in, out, err = popen3('nroff -man')

HOWEVER, this doesn't even work because 'in' is a keyword in Ruby!
That, IMHO, should be fixed. (Both this specific case and documentation
for libraries in general. However, I obviously don't care *that* much,
otherwise I'd be writing documentation right now.)

Jason Creighton

[Gavin Sinclair]

>> This points up my major frustration with Ruby-- I had to completely

>> guess as to how Open3 worked, or *even that it existed at all*. There
>> are no docs I can find for it, and this makes developing with Ruby
>> more of a pain than it needs to be. If I were doing this in perl, I'd
>> just have to 'perldoc -q pipe', and instantly I have docs on
>> IPC::Open2 and a reference to IPC::Open3 if I need that.

>
> Yes, you're right, the documentation should be better. Of course, if you
> look at open3.rb, you'll see this:
>
> # Usage:
> # require "open3"
> #
> # in, out, err = Open3.popen3('nroff -man')
> # or
> # include Open3
> # in, out, err = popen3('nroff -man')
>
> HOWEVER, this doesn't even work because 'in' is a keyword in Ruby! That,
> IMHO, should be fixed. (Both this specific case and documentation for
> libraries in general. However, I obviously don't care *that* much,
> otherwise I'd be writing documentation right now.)
>
> Jason Creighton

[Gavin Sinclair]

>>> This points up my major frustration with Ruby-- I had to completely

>>> guess as to how Open3 worked, or *even that it existed at all*.
>>> There are no docs I can find for it, and this makes developing with
>>> Ruby more of a pain than it needs to be. If I were doing this in
>>> perl, I'd just have to 'perldoc -q pipe', and instantly I have docs
>>> on
>>> IPC::Open2 and a reference to IPC::Open3 if I need that.

>>
>> Yes, you're right, the documentation should be better. Of course, if
>> you look at open3.rb, you'll see this:
>>
>> # Usage:
>> # require "open3"
>> #
>> # in, out, err = Open3.popen3('nroff -man')
>> # or
>> # include Open3
>> # in, out, err = popen3('nroff -man')
>>
>> HOWEVER, this doesn't even work because 'in' is a keyword in Ruby!
>> That, IMHO, should be fixed. (Both this specific case and
>> documentation for libraries in general. However, I obviously don't
>> care *that* much, otherwise I'd be writing documentation right now.)
>>
>> Jason Creighton

Damn %#$^#$ web email client! %^#@$%@#$

Sorry for the redundant post. What I was going to say is:

Here's how you can ensure this is fixed:
* subscribe to [email]ruby-core@ruby-lang.org[/email]
* create a patch to fix that documentation issue (diff -u format, I think)
* send it to ruby-core, as a "PATCH: ..." subject

It seems heavyweight, but it makes it easier for the maintainer to go
"yep, that looks good, I'll commit that". And besides, once you've
subscribed, it's easier to contribute more patches!

Of course, for some files, it may be better to send patches directly to
the author, but if the file is in the Ruby distro, ruby-core is a good
catch-all.

Gavin

[Thomas Adam]

--- Gavin Sinclair <gsinclair@soyabean.com.au> wrote:

> >>> This points up my major frustration with Ruby-- I had to completely
> >>> guess as to how Open3 worked, or *even that it existed at all*.
> >>> There are no docs I can find for it, and this makes developing with
> >>> Ruby more of a pain than it needs to be. If I were doing this in
> >>> perl, I'd just have to 'perldoc -q pipe', and instantly I have docs
> >>> on
> >>> IPC::Open2 and a reference to IPC::Open3 if I need that.
> >>
> >> Yes, you're right, the documentation should be better. Of course, if
> >> you look at open3.rb, you'll see this:
> >>
> >> # Usage:
> >> # require "open3"
> >> #
> >> # in, out, err = Open3.popen3('nroff -man')
> >> # or
> >> # include Open3
> >> # in, out, err = popen3('nroff -man')
> >>
> >> HOWEVER, this doesn't even work because 'in' is a keyword in Ruby!
> >> That, IMHO, should be fixed. (Both this specific case and
> >> documentation for libraries in general. However, I obviously don't
> >> care *that* much, otherwise I'd be writing documentation right now.)
> >>
> >> Jason Creighton

>
> Damn %#$^#$ web email client! %^#@$%@#$
>
> Sorry for the redundant post. What I was going to say is:
>
> Here's how you can ensure this is fixed:
> * subscribe to [email]ruby-core@ruby-lang.org[/email]
> * create a patch to fix that documentation issue (diff -u format, I
> think)

Nitpick: unified diffs here, although shorter, can often less, umm,
informative and unless specifically asked for, I would suggest sending in
both a non-unified and a unified diff.

Also, it is not a patch file, until it has been applied :)

> * send it to ruby-core, as a "PATCH: ..." subject
>
> It seems heavyweight, but it makes it easier for the maintainer to go
> "yep, that looks good, I'll commit that". And besides, once you've
> subscribed, it's easier to contribute more patches!

Given Jason's busy workload (I would imagine) I doubt he will be able to
contribute that many (if any) patches.

> Of course, for some files, it may be better to send patches directly to
> the author, but if the file is in the Ruby distro, ruby-core is a good
> catch-all.

Since you are such an advocate in creating patches and the like, perhaps
you would send one in? :)

-- Thomas Adam

=====
Thomas Adam

"The Linux Weekend Mechanic" -- [url]www.linuxgazette.com[/url]

__________________________________________________ ______________________
Want to chat instantly with your online friends? Get the FREE Yahoo!
Messenger [url]http://mail.messenger.yahoo.co.uk[/url]

[Robert Klemme]

"Thomas Adam" <thomas_adam16@yahoo.com> schrieb im Newsbeitrag
news:20030911081203.29175.qmail@web41104.mail.yaho o.com...

> Nitpick: unified diffs here, although shorter, can often less, umm,
> informative and unless specifically asked for, I would suggest sending

in

> both a non-unified and a unified diff.

Curions why you say that: I find unified diffs the most informative ones
for a human reader. And if -u doesn't suit you, you can use -u<n> where n
denotes a larger context. Just my 2 cent...

Regards

robert

[Gavin Sinclair]

>> Here's how you can ensure this is fixed:
>> * subscribe to [email]ruby-core@ruby-lang.org[/email]
>> * create a patch to fix that documentation issue (diff -u format, I
>> think)

>
> Nitpick: unified diffs here, although shorter, can often less, umm,
> informative and unless specifically asked for, I would suggest sending
> in both a non-unified and a unified diff.

I've never seen two diffs sent before, and I believe the standard on
ruby-core (I'm not a licenced representative, BTW :) is unified. I've
seen a complaint when a plain diff was sent.

> Also, it is not a patch file, until it has been applied :)

That's a new one :/

Isn't the output of 'diff' considered fair game as input to 'patch'?
Don't uber-geeks simply run 'patch' on the entire contents of an email and
trust it to do the right thing?

>> * send it to ruby-core, as a "PATCH: ..." subject
>>
>> It seems heavyweight, but it makes it easier for the maintainer to go
>> "yep, that looks good, I'll commit that". And besides, once you've
>> subscribed, it's easier to contribute more patches!

>
> Given Jason's busy workload (I would imagine) I doubt he will be able to
> contribute that many (if any) patches.

The suggested patch would take less than 5 minutes. It might not seem
like much of an outcome, but the maintainer, on applying it, would
probably be prompted to do some general improvement to the file.

>> Of course, for some files, it may be better to send patches directly
>> to the author, but if the file is in the Ruby distro, ruby-core is a
>> good catch-all.

>
> Since you are such an advocate in creating patches and the like, perhaps
> you would send one in? :)

Well, I'm up to my eyeballs with other Ruby documentation concerns, and
not even achieving that much. And I have no personal concern for popen3,
or whatever the issue is. But I'll see what I can do.

Gavin

[Jason Creighton]

On Thu, 11 Sep 2003 18:23:45 +0900
"Gavin Sinclair" <gsinclair@soyabean.com.au> wrote:

> >> Here's how you can ensure this is fixed:
> >> * subscribe to [email]ruby-core@ruby-lang.org[/email]
> >> * create a patch to fix that documentation issue (diff -u format, I
> >> think)

> >
> > Nitpick: unified diffs here, although shorter, can often less, umm,
> > informative and unless specifically asked for, I would suggest sending
> > in both a non-unified and a unified diff.

I find unified diffs easier to read. What are you suggesting as a
non-unified diff? (Plain old diff? Context diff? What?)

Plus unified diffs are more robust because of the context included.

> I've never seen two diffs sent before, and I believe the standard on
> ruby-core (I'm not a licenced representative, BTW :) is unified. I've
> seen a complaint when a plain diff was sent.
>

> > Also, it is not a patch file, until it has been applied :)

>
> That's a new one :/
>
> Isn't the output of 'diff' considered fair game as input to 'patch'?
> Don't uber-geeks simply run 'patch' on the entire contents of an email and
> trust it to do the right thing?
>

> >> * send it to ruby-core, as a "PATCH: ..." subject
> >>
> >> It seems heavyweight, but it makes it easier for the maintainer to go
> >> "yep, that looks good, I'll commit that". And besides, once you've
> >> subscribed, it's easier to contribute more patches!

> >
> > Given Jason's busy workload (I would imagine)

LOL! When you're not out of high school, there's really not that much
work to be done. :)

> > I doubt he will be able to contribute that many (if any) patches.

>
> The suggested patch would take less than 5 minutes. It might not seem
> like much of an outcome, but the maintainer, on applying it, would
> probably be prompted to do some general improvement to the file.

No, actually, the main thing I'm worried about is subscribing to
ruby-core. About how many messages/day there? Might have to fork some
money over to softhome.net to get a higher mail quota. Anyhow, I think
I'll subscribe to ruby-core and submit the open3.rb doc patch.

<thinking out loud>

It Would Be Nice(tm) if there was an email address that would forward to
ruby-core so people wouldn't have to subscribe for simple little patches
like this. Maybe have to include "ruby-core" in the subject line or
something so spam doesn't get through.

> >> Of course, for some files, it may be better to send patches directly
> >> to the author, but if the file is in the Ruby distro, ruby-core is a
> >> good catch-all.

Yes, especially because there's no author listed for open3.rb. So the
general rule of thumb would be that if it's a package of it's own right
(something like test/unit or webrick or something), sumbit patch to
author and it'll get into Ruby next time he/she does a CVS commit, but
if it's on its own, like open3.rb, send to ruby-core?

Jason Creighton

[Eric Schwartz]

Jason Creighton <androflux@softhome.net.remove.to.reply> writes:

> Yes, you're right, the documentation should be better. Of course, if you
> look at open3.rb, you'll see this:

It's not so much bad documentation as completely nonexistent
documentation. AFAIK, there's no documentation that even lets me know
there is such a thing as Open3. While it's good to be able to look at
the source in case the documentation's ambiguous or misleading, it
shouldn't be a requirement in learning how to use something in the
first place.

-=Eric
--
Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
-- Blair Houghton.

[Gavin Sinclair]

>> >> * send it to ruby-core, as a "PATCH: ..." subject
>> >>
>> >> It seems heavyweight, but it makes it easier for the maintainer to

>> go "yep, that looks good, I'll commit that". And besides, once
>> you've subscribed, it's easier to contribute more patches!

>> >
>> > Given Jason's busy workload (I would imagine)

>
> LOL! When you're not out of high school, there's really not that much
> work to be done. :)

Sigh... those were the days!

>> > I doubt he will be able to contribute that many (if any) patches.

>>
>> The suggested patch would take less than 5 minutes. It might not seem
>> like much of an outcome, but the maintainer, on applying it, would
>> probably be prompted to do some general improvement to the file.

>
> No, actually, the main thing I'm worried about is subscribing to
> ruby-core. About how many messages/day there? Might have to fork some
> money over to softhome.net to get a higher mail quota. Anyhow, I think
> I'll subscribe to ruby-core and submit the open3.rb doc patch.

It's a low-volume list, probably averages 5 emails a day at most.

> <thinking out loud>
>
> It Would Be Nice(tm) if there was an email address that would forward to
> ruby-core so people wouldn't have to subscribe for simple little patches
> like this. Maybe have to include "ruby-core" in the subject line or
> something so spam doesn't get through.

I don't know if you even have to be subscribed to send stuff there but you
would probably want to see replies to your post anyway.

>> >> Of course, for some files, it may be better to send patches

>> directly to the author, but if the file is in the Ruby distro,
>> ruby-core is a good catch-all.

>
> Yes, especially because there's no author listed for open3.rb. So the
> general rule of thumb would be that if it's a package of it's own right
> (something like test/unit or webrick or something), sumbit patch to
> author and it'll get into Ruby next time he/she does a CVS commit, but
> if it's on its own, like open3.rb, send to ruby-core?

Well, ruby-core is a good catch-all, as most if not all package
maintainers would be subscribed. And someone else could commit it on
their behalf anyway. It might be best to target authors to avoid
overloading ruby-core, but I don't think that'll be a problem just yet!
You common sense will be fine.

I may be way off in my advice, but nobody has offered anything else :)

> Jason Creighton

Cheers,
Gavin