Flocking Advise

[\Dandy\ Randy]

Hey ... could use some advise on the best file locking method ... at present
I use the following:

#!/usr/bin/perl

use strict;
use 5.004;
use Fcntl qw(:DEFAULT :flock);

open (FH, "<data.txt") or die "Can't open file: $!";
flock (FH, LOCK_EX) or die "Can't lock file: $!";
$data=<FH>;
chomp ($data);
($total,$opened,$followed)=split(/\|/,$data);
close(FH);

$opened = $opened + 1;

sysopen(FH, "data.txt", O_WRONLY | O_CREAT) or die "can't open filename:
$!";
flock (FH, LOCK_EX) or die "can't lock filename: $!";
truncate (FH, 0) or die "can't truncate filename: $!";
print FH "$total|$opened|$followed\n";
close FH;

print "Content-type: text/html \n\n";
print "Done\n";
exit;

It seems to work ok, but am not sure if it is a good approach or not. One of
my main questions is ... what about sememorph locks ... been reading in
different places that you start you file open code by opening a dummy
sememorph file ... kinda like this:

open S, "> somefile.sem" or die ...;
flock S, LOCK_EX or die ...;
open F, "> somefile" or die ...;
# Now write F
close F;
close S;

What I see going on here is the code opens a file that does not contain
critical data, thus protecting the real data, and the user cannot open the
real file until the sem lock becomes free ... Two questions ... is this a
good approach, and secondly ... whats inside the sem file? is it an empty
dummy file? does it have to have a .sem extension? Thank you for you
opinions.

Randy

[John Bokma]

"Dandy" Randy wrote:

> Hey ... could use some advise on the best file locking method ... at present
> I use the following:
>
> #!/usr/bin/perl
>
> use strict;
> use 5.004;
> use Fcntl qw(:DEFAULT :flock);
>
> open (FH, "<data.txt") or die "Can't open file: $!";
> flock (FH, LOCK_EX) or die "Can't lock file: $!";
> $data=<FH>;
> chomp ($data);
> ($total,$opened,$followed)=split(/\|/,$data);
> close(FH);
>
> $opened = $opened + 1;

BEEP *RED ALERT* RACE CONDITION DETECTED....

suppose the script is called in between opens the file and writes the
new counter value and then....

> sysopen(FH, "data.txt", O_WRONLY | O_CREAT) or die "can't open filename:
> $!";
> flock (FH, LOCK_EX) or die "can't lock filename: $!";
> truncate (FH, 0) or die "can't truncate filename: $!";
> print FH "$total|$opened|$followed\n";
> close FH;

hell breaks lose since you just overwrote the counter....

eg:

a -> reads 5
b -> reads 5
b -> increments
b -> writes 6
a -> increments
a -> writes 6

On a busy site there could be a, b, c etc.

> What I see going on here is the code opens a file that does not contain
> critical data, thus protecting the real data, and the user cannot open the
> real file until the sem lock becomes free ... Two questions ... is this a
> good approach, and secondly ... whats inside the sem file? is it an empty
> dummy file? does it have to have a .sem extension? Thank you for you
> opinions.

The lock on your file is *the* semaphore. No need for a separate lock file.


--
Kind regards, feel free to mail: mail(at)johnbokma.com (or reply)
virtual home: [url]http://johnbokma.com/[/url] ICQ: 218175426
John web site hints: [url]http://johnbokma.com/websitedesign/[/url]

[David Efflandt]

On Tue, 09 Sep 2003 01:03:25 GMT, \"Dandy\" Randy <ducott@hotmail.com> wrote:

> Hey ... could use some advise on the best file locking method ... at present
> I use the following:
>
> #!/usr/bin/perl
>
> use strict;
> use 5.004;
> use Fcntl qw(:DEFAULT :flock);
>
> open (FH, "<data.txt") or die "Can't open file: $!";
> flock (FH, LOCK_EX) or die "Can't lock file: $!";
> $data=<FH>;
> chomp ($data);
> ($total,$opened,$followed)=split(/\|/,$data);
> close(FH);
>
> $opened = $opened + 1;
>
> sysopen(FH, "data.txt", O_WRONLY | O_CREAT) or die "can't open filename:
> $!";
> flock (FH, LOCK_EX) or die "can't lock filename: $!";
> truncate (FH, 0) or die "can't truncate filename: $!";
> print FH "$total|$opened|$followed\n";
> close FH;
>
> print "Content-type: text/html \n\n";
> print "Done\n";
> exit;
>
> It seems to work ok, but am not sure if it is a good approach or not. One of
> my main questions is ... what about sememorph locks ... been reading in
> different places that you start you file open code by opening a dummy
> sememorph file ... kinda like this:

One problem with what you are doing (hit counter?) is, what happens if
multiple instances of the script run at the same time? One script reads
it, closes it, another script opens it and reads it, first script opens
and writes to it, and second script modifies stale data and saves it.

My approach is to open it read/write ( +< ), flock it, seek the beginning
(just in case something else opened it before flock), modify and save
data, and then close it. Then another instance would have to wait until
flock cleared before it was able to flock, read and modify fresh data.
Otherwise your count could easily end up short.

--
David Efflandt - All spam ignored [url]http://www.de-srv.com/[/url]
[url]http://www.autox.chicago.il.us/[/url] [url]http://www.berniesfloral.net/[/url]
[url]http://cgi-help.virtualave.net/[/url] [url]http://hammer.prohosting.com/~cgi-wiz/[/url]

[news@roaima.freeserve.co.uk]

David Efflandt <efflandt@xnet.com> wrote:

> My approach is to open it read/write ( +< ), flock it, seek the beginning
> (just in case something else opened it before flock), modify and save
> data, and then close it.

There's a very similar issue going on in a parallel thread at the moment.

What I really don't see is why there's a requirement to seek back to the
beginning of a file after it's been flocked. If you've not read anything
then the file pointer's at the start of the file anyway. (flock applies a
file lock rather than a partial region lock.) I appreciate you need a seek
when swapping between reading and writing, but that's not the issue here.

Can anyone give a concrete explanation as to why the seek is required
immediately after the flock?

Cheers,
Chris
--
@s=split(//,"Je,\nhn ersloak rcet thuarP");$k=$l=@s;for(;$k;$k--){$i=($i+1)%$l
until$s[$i];$c=$s[$i];print$c;undef$s[$i];$i=($i+(ord$c))%$l}

[David Efflandt]

On Tue, 9 Sep 2003, [email]news@roaima.freeserve.co.uk[/email]
<news@roaima.freeserve.co.uk> wrote:

> David Efflandt <efflandt@xnet.com> wrote:

>> My approach is to open it read/write ( +< ), flock it, seek the beginning
>> (just in case something else opened it before flock), modify and save
>> data, and then close it.

>
> There's a very similar issue going on in a parallel thread at the moment.
>
> What I really don't see is why there's a requirement to seek back to the
> beginning of a file after it's been flocked. If you've not read anything
> then the file pointer's at the start of the file anyway. (flock applies a
> file lock rather than a partial region lock.) I appreciate you need a seek
> when swapping between reading and writing, but that's not the issue here.
>
> Can anyone give a concrete explanation as to why the seek is required
> immediately after the flock?

perldoc -f flock

Basically something else could open the file (possibly for reading)
between the time you open it, and flock it, so the file pointer might not
be at the beginning of the file when you go to read it. It may not always
be necessary, but it is playing it safe, just in case.

--
David Efflandt - All spam ignored [url]http://www.de-srv.com/[/url]

[ctcgag@hotmail.com]

"\"Dandy\" Randy" <ducott@hotmail.com> wrote:

> Hey ... could use some advise on the best file locking method ... at
> present I use the following:
>
> #!/usr/bin/perl
>
> use strict;
> use 5.004;
> use Fcntl qw(:DEFAULT :flock);
>
> open (FH, "<data.txt") or die "Can't open file: $!";
> flock (FH, LOCK_EX) or die "Can't lock file: $!";
> $data=<FH>;
> chomp ($data);
> ($total,$opened,$followed)=split(/\|/,$data);
> close(FH);

Bang, you're dead! There was no point taking out an exclusive lock,
since you just lost it before the write occured.

> $opened = $opened + 1;
>
> sysopen(FH, "data.txt", O_WRONLY | O_CREAT) or die "can't open filename:
> $!";
> flock (FH, LOCK_EX) or die "can't lock filename: $!";
> truncate (FH, 0) or die "can't truncate filename: $!";
> print FH "$total|$opened|$followed\n";
> close FH;
>
> print "Content-type: text/html \n\n";
> print "Done\n";
> exit;
>
> It seems to work ok, but am not sure if it is a good approach or not.

It is not. Both operations need to take place under the *same lock*,
not under two different locks. If it seems to work, you are probably
not testing under sufficiently contentious conditions.

> One
> of my main questions is ... what about sememorph locks ... been reading
> in different places that you start you file open code by opening a dummy
> sememorph file ... kinda like this:

You do not need to do this if flock works properly on your system.

One things you could do is open the file for read and write,
obtain the EX lock, read, truncate, write, close.

Another way is to use the file as it's own semaphore:
open FH for reading, obtain EX lock, read. *Do not close FH*
open FH2 for writing to the same file. (There is no need to lock
it as FH holds the lock on FH2's behalf). Write to FH2, close
FH2, close FH.

This last is the way I usually do it, as I've never bothered to
learn how to use truncate and +<.


Xho

--
-------------------- [url]http://NewsReader.Com/[/url] --------------------
Usenet Newsgroup Service New Rate! $9.95/Month 50GB

[news@roaima.freeserve.co.uk]

<news@roaima.freeserve.co.uk> asked:

> Can anyone give a concrete explanation as to why the seek is required
> immediately after the flock?

David Efflandt <efflandt@xnet.com> wrote:

> perldoc -f flock

The seek to EOF is correct when you're appending to a file. In the
general case (and particularly when reading) it's not required.

> Basically something else could open the file (possibly for reading)
> between the time you open it, and flock it, so the file pointer might not
> be at the beginning of the file when you go to read it.

The file pointer is on a per-filehandle basis, not on a per-file
basis. We'd get mighty upset if I changed my file pointer and yours
updated too.

The notable exception is when *writing*, where we've specified that all
writing is to occur at the current EOF (i.e. append), in which case it's
correct behaviour).

Chris
--
@s=split(//,"Je,\nhn ersloak rcet thuarP");$k=$l=@s;for(;$k;$k--){$i=($i+1)%$l
until$s[$i];$c=$s[$i];print$c;undef$s[$i];$i=($i+(ord$c))%$l}

[Alan J. Flavell]

On Tue, Sep 9, [email]ctcgag@hotmail.com[/email] inscribed on the eternal scroll:

> One things you could do is open the file for read and write,
> obtain the EX lock, read, truncate, write, close.

This is an FAQ, isn't it? It always makes me nervous to see
independent proposals offered to the readership when there's an
existing FAQ that addresses their requirement. If the FAQ is wrong or
incomplete or otherwise unsatisfactory, it seems to me that discussion
should focus on a proposal for a revised FAQ, rather than offering the
readership some kind of parallel-universe advice. No offence meant.

> Another way is to use the file as it's own semaphore:
> open FH for reading, obtain EX lock, read. *Do not close FH*
> open FH2 for writing to the same file. (There is no need to lock
> it as FH holds the lock on FH2's behalf). Write to FH2, close
> FH2, close FH.

But now we're going in circles, because in a very recent posting on
this topic a contributor pointed out - and the FAQ says so too - that
on some platforms you can't get an exclusive lock without opening for
write.

See also the file-locking part of perlopentut (again, if there's
something wrong, the discussion should IMHO be contributing to fix it,
not just talking across it...)

> This last is the way I usually do it, as I've never bothered to
> learn how to use truncate and +<.

Well, I can only say that according to the available documentation,
your method appears to be less portable. OK, OK, as the FAQ says:

| Slavish adherence to portability concerns shouldn't get in the way
| of your getting your job done.

but where there's a choice, I'd try to incline to the portable one.

best regards

[url]http://www.perldoc.com/perl5.8.0/pod/perlfaq5.html#How-can-I-lock-a-file-[/url]
[url]http://www.perldoc.com/perl5.8.0/pod/perlopentut.html#File-Locking[/url]

[David Efflandt]

On Tue, 9 Sep 2003, [email]news@roaima.freeserve.co.uk[/email]
<news@roaima.freeserve.co.uk> wrote:

><news@roaima.freeserve.co.uk> asked:

>> Can anyone give a concrete explanation as to why the seek is required
>> immediately after the flock?

>
> David Efflandt <efflandt@xnet.com> wrote:

>> perldoc -f flock

>
> The seek to EOF is correct when you're appending to a file. In the
> general case (and particularly when reading) it's not required.
>

>> Basically something else could open the file (possibly for reading)
>> between the time you open it, and flock it, so the file pointer might not
>> be at the beginning of the file when you go to read it.

>
> The file pointer is on a per-filehandle basis, not on a per-file
> basis. We'd get mighty upset if I changed my file pointer and yours
> updated too.

Actually I had problems with a CGI script that just read a file without
flocking it. If more than one instance tried to read from the file at the
same time, it apparently got confused where the file pointer was and the
Perl (CGI) scripts would go into endless loops (the host notified me about
the endless processes). Flocking solved that. I think that was on
Solaris, which may do things differently than other Unix systems.

--
David Efflandt - All spam ignored [url]http://www.de-srv.com/[/url]
[url]http://www.autox.chicago.il.us/[/url] [url]http://www.berniesfloral.net/[/url]
[url]http://cgi-help.virtualave.net/[/url] [url]http://hammer.prohosting.com/~cgi-wiz/[/url]

[news@roaima.freeserve.co.uk]

David Efflandt <efflandt@xnet.com> wrote:

> Actually I had problems with a CGI script that just read a file without
> flocking it. If more than one instance tried to read from the file at the
> same time, it apparently got confused where the file pointer was and the
> Perl (CGI) scripts would go into endless loops (the host notified me about
> the endless processes).

That'll be something to do with the way CGI processes have been
implemented in your web server.

Chris
--
@s=split(//,"Je,\nhn ersloak rcet thuarP");$k=$l=@s;for(;$k;$k--){$i=($i+1)%$l
until$s[$i];$c=$s[$i];print$c;undef$s[$i];$i=($i+(ord$c))%$l}