IO#clone and 1.6 -> 1.8 question

[dblack@superlink.net]

Hi --

Just tinkering with IO#clone, in connection with some irc
chatting, and found the following behavior which puzzles
me. Here's the test program:

fh = File.new("nums") # 'nums' is 1..20, one integer per line
puts fh.gets

fd = fh.clone
puts fd.gets

puts fh.gets

and here's output from 1.6.8 and 1.8.0:

ruby 1.6.8 (2002-12-16) [i686-linux]
1
nil
2

ruby 1.8.0 (2003-08-04) [i686-linux]
1
2
nil

Actually I don't understand either of these; I thought that both runs
would produce 1\n2\n3\n.

Can anyone shed light on these examples?


David

--
David Alan Black
home: [email]dblack@superlink.net[/email]
work: [email]blackdav@shu.edu[/email]
Web: [url]http://pirate.shu.edu/~blackdav[/url]

[Yukihiro Matsumoto]

Hi,

In message "IO#clone and 1.6 -> 1.8 question"
on 03/09/09, [email]dblack@superlink.net[/email] <dblack@superlink.net> writes:

|Just tinkering with IO#clone, in connection with some irc
|chatting, and found the following behavior which puzzles
|me. Here's the test program:

It's combination of the two:

* file position is shared among IO clones. It's defined stdio
behavior.

* IO are buffered so that actual file pointer
may advance more than you expected (i.e. entire "num" file was
swallowed this case) by a single "gets".

matz.

[dblack@superlink.net]

Hi --

On Tue, 9 Sep 2003, Yukihiro Matsumoto wrote:

> Hi,
>
> In message "IO#clone and 1.6 -> 1.8 question"
> on 03/09/09, [email]dblack@superlink.net[/email] <dblack@superlink.net> writes:
>
> |Just tinkering with IO#clone, in connection with some irc
> |chatting, and found the following behavior which puzzles
> |me. Here's the test program:
>
> It's combination of the two:
>
> * file position is shared among IO clones. It's defined stdio
> behavior.

That's what I was expecting, but:

fh = File.new("nums")
fd = fh.clone
fh.gets
puts fh.pos
puts fd.pos

=>
2
0

> * IO are buffered so that actual file pointer
> may advance more than you expected (i.e. entire "num" file was
> swallowed this case) by a single "gets".

I'm really puzzled by that. If that's the case, how is it possible to
iterate through the lines of a file? Doesn't it have to be guaranteed
that gets will return the next line?

Also, the thing I'm seeing only seems to happen when a file handle is
cloned. If I just do:

fh = File.new("nums")
20.times { puts fh.gets }

I always get 20 lines of output. But when I clone fh, I start to get
the behavior where one fh advances to EOF and the other doesn't. I'm
afraid I'm still not seeing why cloning a file handle would or should
cause any of this to happen.


David

--
David Alan Black
home: [email]dblack@superlink.net[/email]
work: [email]blackdav@shu.edu[/email]
Web: [url]http://pirate.shu.edu/~blackdav[/url]

[ts]

>>>>> "d" == dblack <dblack@superlink.net> writes:

d> I always get 20 lines of output. But when I clone fh, I start to get
d> the behavior where one fh advances to EOF and the other doesn't. I'm
d> afraid I'm still not seeing why cloning a file handle would or should
d> cause any of this to happen.

With your original example

d> fh = File.new("nums") # 'nums' is 1..20, one integer per line
d> puts fh.gets
d>
d> fd = fh.clone
d> puts fd.gets
d>
d> puts fh.gets

it do this (well with some more tests :-)))

svg% cat nums
1
2
3
4
5
6
7
8
9
svg%

svg% cat a.c
#include <stdio.h>

main()
{
FILE *f1, *f2;
char line[1024];

f1 = fopen("nums", "r");
line[0] = line[1] = line[2] = 0;
fread(line, 1, 2, f1);
printf("%s", line);
fseeko(f1, 0L, SEEK_CUR);
f2 = fdopen(dup(fileno(f1)), "r");
line[0] = line[1] = 0;
fread(line, 1, 2, f2);
printf("%s", line);
line[0] = line[1] = 0;
if (fread(line, 1, 2, f1) == 0) {
printf("EOF\n");
}
else {
printf("%s", line);
}
}
svg%

svg% a.out
1
2
EOF
svg%


Guy Decoux

[Yukihiro Matsumoto]

Hi,

In message "Re: IO#clone and 1.6 -> 1.8 question"
on 03/09/09, [email]dblack@superlink.net[/email] <dblack@superlink.net> writes:

|> It's combination of the two:
|>
|> * file position is shared among IO clones. It's defined stdio
|> behavior.
|
|That's what I was expecting, but:
|
| fh = File.new("nums")
| fd = fh.clone
| fh.gets
| puts fh.pos
| puts fd.pos
|
| =>
| 2
| 0

Ah, I have to tell you something. There's two file positions for an
IO. One is a file position for a file descriptor. This is shared
among duped file descriptors. The other is a file descriptor for a
stdio (i.e. FILE*), this is *not* shared. For example, say we name
the former position as rpos (stands for real position), and the latter
as bpos (stands for buffered position).

When you have "1\n2\n3\n" in a file "num", and you get an IO by
opening it.

fh = File.new("nums")

Its rpos and bpos are 0. You clone fh.

fd = fh.clone

Naturally, clone's rpos and bpos are 0 too. Then you call gets on the
original IO.

fh.gets

It returns "1\n", bpos is right after the first newline, but rpos is
at the end of file, since whole file is read in the fh's buffer.

puts fd.pos

"pos" method returns IO's bpos, so that it returns 0. When you try to
read from fd, since its internal buffer is empty, it tries to read
from the file descriptor. Its rpos is at the end of file (remember
rpos is shared). Thus "gets" returns nil. Weird? I agree, but it's
how stdio defined to work.

So what this means? It means mixing buffered read with shared file
descriptor cause real confusion, just like the one you face. Do not
use buffered read with IO clones. Use sysread if you want to work
with IO clones.

matz.

[ts]

>>>>> "d" == dblack <dblack@superlink.net> writes:

d> Also, the thing I'm seeing only seems to happen when a file handle is
d> cloned. If I just do:

Well, if you have access to a Solaris system read carefully stdio(3s), you
have

[...]
Handles can be created or destroyed by user action without
affecting the underlying open file description. Some of the
ways to create them include fcntl(2), dup(2), fdopen(3S),
[...]
If two or more handles are used, and any one of them is a
stream, their actions shall be coordinated as described
below. If this is not done, the result is undefined.
[...]
10.
For the second handle: if any previous active handle
has called a function that explicitly changed the file
offset, except as required above for the first handle,
the application shall perform an lseek() or an
fseek(2) (as appropriate to the type of the handle) to
an appropriate location.
[...]


Guy Decoux