Defeating OS buffering?

[Eric Schwartz]

I'm trying to use Perl to automate a specific test process, but OS
buffering is defeating some of the main utility of it. I run a
program like this:

open(TEST, "$testcmd |") or die "bah-- $!";
while(<TEST>) {
# do stuff
}
close(TEST);

Part of the problem here is that the test referred to in $testcmd is a
bash script that execs a C program, which I do not have source to.
This C program doesn't print a lot of output, so as a consequence, I
don't see any of it until the C program exits. Is there any way,
short of modifying the C program, that I can defeat the OS buffering
of the C program's output?

-=Eric, fully expecting the answer to be "no". :(
--
Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
-- Blair Houghton.

[Vlad Tepes]

Eric Schwartz <emschwar@pobox.com> wrote:

> I'm trying to use Perl to automate a specific test process, but OS
> buffering is defeating some of the main utility of it. I run a
> program like this:
>
> open(TEST, "$testcmd |") or die "bah-- $!";
> while(<TEST>) {
> # do stuff
> }
> close(TEST);
>
> Part of the problem here is that the test referred to in $testcmd is a
> bash script that execs a C program, which I do not have source to.
> This C program doesn't print a lot of output, so as a consequence, I
> don't see any of it until the C program exits. Is there any way,
> short of modifying the C program, that I can defeat the OS buffering
> of the C program's output?
>
> -=Eric, fully expecting the answer to be "no". :(

How about reading single bytes bytes from the program?

open(TEST, "/home/t/c/a.out |") or die "bah-- $!";
$|++;
while( 0 != read TEST, $_, 1, 1 ) {
print;
}
close(TEST);

HTH,
--
Vlad

[Eric Schwartz]

Vlad Tepes <minceme@start.no> writes:

> How about reading single bytes bytes from the program?
>
> open(TEST, "/home/t/c/a.out |") or die "bah-- $!";

To reproduce this properly, you'd need a bash program that just did:

exec /home/t/c/a.out

and you'd invoke the bash program there.

> $|++;

This unbuffers STDOUT, which doesn't help when the problem is that the
OS is buffering the TEST filehandle.

> while( 0 != read TEST, $_, 1, 1 ) {
> print;
> }
> close(TEST);

How is reading input that doesn't show up a character at a time going
to help? Either it's there, or it isn't; reading it in smaller chunks
is just making things more inefficient without actually changing
anything.

> HTH,

I'm afraid not. Thanks for the effort, anyhow.

-=Eric
--
Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
-- Blair Houghton.

[Vlad Tepes]

Eric Schwartz <emschwar@pobox.com> wrote:

> Vlad Tepes <minceme@start.no> writes:

>

>> open(TEST, "/home/t/c/a.out |") or die "bah-- $!";

>
> To reproduce this properly, you'd need a bash program that just did:
>
> exec /home/t/c/a.out

Ok, now I have created a bash-script that exec'ed the a.out, and used
that in the open-statement above. It worked nicely, doing what you
wanted.

Maybe I'm missing something?

The script I used:

#!/bin/bash
exec "/home/t/c/a.out";

The C-program:

#include <stdio.h>
#include <unistd.h>

int main ( void ) {
printf("Some text...");
fflush(NULL);
sleep(3);
printf("some more text...\n");
return(0);
}

The perl program:

#!/usr/bin/perl
use warnings;
use strict;

open(TEST, "/tmp/t.sh |") or die "bah-- $!";
$|++;
while( 0 != read TEST, $_, 1, 1 ){
print;
}
close(TEST);

Hope this helps,
--
Vlad

[Trent Curry]

"Vlad Tepes" <minceme@start.no> wrote in message
news:bjo0fn$8gf$1@troll.powertech.no...

> Eric Schwartz <emschwar@pobox.com> wrote:

> > Vlad Tepes <minceme@start.no> writes:

>

[Other code]

> The perl program:
>
> #!/usr/bin/perl
> use warnings;
> use strict;
>
> open(TEST, "/tmp/t.sh |") or die "bah-- $!";
> $|++;
> while( 0 != read TEST, $_, 1, 1 ){
> print;
> }
> close(TEST);

Or, as an FYI, you could rewrite the while as:

print while (0 != read TEST, $_, 1, 1);

[Trent Curry]

"Eric Schwartz" <emschwar@pobox.com> wrote in message
news:etod6e8zbl2.fsf@wormtongue.emschwar...

> Vlad Tepes <minceme@start.no> writes:

> > How about reading single bytes bytes from the program?
> >
> > open(TEST, "/home/t/c/a.out |") or die "bah-- $!";

>
> To reproduce this properly, you'd need a bash program that just did:
>
> exec /home/t/c/a.out
>
> and you'd invoke the bash program there.
>

> > $|++;

>
> This unbuffers STDOUT, which doesn't help when the problem is that the
> OS is buffering the TEST filehandle.

From perldoc perlvar:

$OUTPUT_AUTOFLUSH
$|

If set to nonzero, forces a flush right away and after every write
or print on the currently selected output channel. [Snipped rest]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So in this case, TEST is the currently selected output handle.

[Eric Schwartz]

Vlad Tepes <minceme@start.no> writes:

<snip>

> The C-program:
>
> #include <stdio.h>
> #include <unistd.h>
>
> int main ( void ) {
> printf("Some text...");
> fflush(NULL);

^^^^^^^^^^^^^
<snip>

That's the problem. Take that out, and it won't work. As I said, I
don't have control over the C program, so I can't make it fflush.

-=Eric
--
Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
-- Blair Houghton.

[Eric Schwartz]

"Trent Curry" <tcurrey@no.no.no.i.said.no> writes:

> "Eric Schwartz" <emschwar@pobox.com> wrote in message
> news:etod6e8zbl2.fsf@wormtongue.emschwar...

>> Vlad Tepes <minceme@start.no> writes:

>> > $|++;

>>
>> This unbuffers STDOUT, which doesn't help when the problem is that the
>> OS is buffering the TEST filehandle.

>
> From perldoc perlvar:
>
> $OUTPUT_AUTOFLUSH
> $|
>
> If set to nonzero, forces a flush right away and after every write
> or print on the currently selected output channel. [Snipped rest]
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> So in this case, TEST is the currently selected output handle.

No, TEST is open for *INPUT*. Hint: if you can read from it, it's
open for input. If you can write to it, it's open for output. And
even if it were open for output, $| only works on filehandles that you
select with the one-arg select(), which Vlad did not do.

-=Eric
--
Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
-- Blair Houghton.

[Tad McClellan]

Trent Curry <tcurrey@no.no.no.i.said.no> wrote:

> So in this case, TEST is the currently selected output handle.

Not unless there was a call to select() missing from the
code that was posted.

Why go on about output when the OP's problem is with _in_put?


--
Tad McClellan SGML consulting
[email]tadmc@augustmail.com[/email] Perl programming
Fort Worth, Texas

[Trent Curry]

"Eric Schwartz" <emschwar@pobox.com> wrote in message
news:etowucgxrim.fsf@wormtongue.emschwar...

> "Trent Curry" <tcurrey@no.no.no.i.said.no> writes:

> > "Eric Schwartz" <emschwar@pobox.com> wrote in message
> > news:etod6e8zbl2.fsf@wormtongue.emschwar...

> >> Vlad Tepes <minceme@start.no> writes:
> >> > $|++;
> >>
> >> This unbuffers STDOUT, which doesn't help when the problem is that the
> >> OS is buffering the TEST filehandle.

> >
> > From perldoc perlvar:
> >
> > $OUTPUT_AUTOFLUSH
> > $|
> >
> > If set to nonzero, forces a flush right away and after every write
> > or print on the currently selected output channel. [Snipped rest]
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
> > So in this case, TEST is the currently selected output handle.

>
> No, TEST is open for *INPUT*. Hint: if you can read from it, it's
> open for input. If you can write to it, it's open for output. And
> even if it were open for output, $| only works on filehandles that you
> select with the one-arg select(), which Vlad did not do.

Sorry, you are right, I over looked that little but very significant part.
Thanks for catching that.

[Trent Curry]

"Tad McClellan" <tadmc@augustmail.com> wrote in message
news:slrnblv8km.58r.tadmc@magna.augustmail.com...

> Trent Curry <tcurrey@no.no.no.i.said.no> wrote:
>
>

> > So in this case, TEST is the currently selected output handle.

>
>
> Not unless there was a call to select() missing from the
> code that was posted.
>
> Why go on about output when the OP's problem is with _in_put?

Sorry, that was my screw up. Over looked that part. My apologies.

[Vlad Tepes]

Eric Schwartz <emschwar@pobox.com> wrote:

> Vlad Tepes <minceme@start.no> writes:
>
> <snip>
>

>> The C-program:
>>
>> #include <stdio.h>
>> #include <unistd.h>
>>
>> int main ( void ) {
>> printf("Some text...");
>> fflush(NULL);

> ^^^^^^^^^^^^^
> <snip>
>
> That's the problem. Take that out, and it won't work. As I said, I
> don't have control over the C program, so I can't make it fflush.

Then I have to say pass. :-(

I don't know if it is possible to force a program to flush it's output.
And if it is, I suppose the solution will be OS-dependant.

--
Vlad

[Kevin Michael Vail]

In article <bjokeh$fat$2@troll.powertech.no>,
Vlad Tepes <minceme@start.no> wrote:

> Eric Schwartz <emschwar@pobox.com> wrote:
>

> > Vlad Tepes <minceme@start.no> writes:
> >
> > <snip>
> >

> >> The C-program:
> >>
> >> #include <stdio.h>
> >> #include <unistd.h>
> >>
> >> int main ( void ) {
> >> printf("Some text...");
> >> fflush(NULL);

> > ^^^^^^^^^^^^^
> > <snip>
> >
> > That's the problem. Take that out, and it won't work. As I said, I
> > don't have control over the C program, so I can't make it fflush.

>
> Then I have to say pass. :-(
>
> I don't know if it is possible to force a program to flush it's output.
> And if it is, I suppose the solution will be OS-dependant.

Not that this is going to be useful, but if you can put a ptty between
the parent and the child, the child should revert to line buffering
because it'll think its output is going to a device. I have no idea how
to go about this in Perl, though (I used to know how to do it in C).
Also, only works under *NIX variants, as far as I know.
--
Kevin Michael Vail | Dogbert: That's circular reasoning.
[email]kevin@vaildc.net[/email] | Dilbert: I prefer to think of it as no loose ends.
[url]http://www.vaildc.net/kevin/[/url]

[nobull@mail.com]

Eric Schwartz <emschwar@pobox.com> wrote in message news:<eto1xuoz66r.fsf@wormtongue.emschwar>...

> Vlad Tepes <minceme@start.no> writes:
>
> <snip>
>

> > The C-program:
> >
> > #include <stdio.h>
> > #include <unistd.h>
> >
> > int main ( void ) {
> > printf("Some text...");
> > fflush(NULL);

> ^^^^^^^^^^^^^
> <snip>
>
> That's the problem. Take that out, and it won't work. As I said, I
> don't have control over the C program, so I can't make it fflush.

It will probably fflush() implicitly on newline if it thinks it's
STDOUT is going to an interactive terminal.

I Expect (hint, hint) you could find something on CPAN that would
allow you to fool a subprocess into thinking it's connected to a
terminal.