logrotate and perl file handles. How do I know when to reopen my log file?

[Teamsolco]

I have written a long running perl application to help me with some trivial administrative functions. Due to the
volume of logging generated by this application, I am managing its output with logrotate. The problem I face is that
perl is "following" the log files that logrotate swaps out. While this is probably intelligent on perl's part (to
follow the file descriptor rather than the file name), it is presenting a major problem for me. When "logfile" becomes
"logfile.1" (etc), my application follows that move and "continues" logging to the new "logfile.1" instead of losing
that file handle -- a condition which I check for so that I can reopen the log file using the proper file name,
"logfile".
Ultimately, perl will only lose the file handle when logrotate finally deletes the "last file" in the log rotation
scheme. Unfortunately, I will have lost a week's worth of logs because all that data will have been in the single file.
Additionally, I'll have several empty log file copies named .1, .2, .3, etc. How can I set perl to NOT follow the file
when it is renamed by logrotate, such that a condition will be created where I will be able to initialize the
replacement file?

Thanks!!

- William Kimball
"Programming is an art-form that fights back!"

[Teamsolco]

I think you have me backwards. I'm not monitoring the log file, I'm writing to it (append mode). Within perl, I need
to know when an outside process has renamed the log file so I can reinitialize the log output to a new file (the correct
file name).

----- Original Message -----
From: "Robby Russell" <rrussell@commandprompt.com>
To: "TeamSolCO" <teamsolco@teamdelsol.com>
Sent: Thursday, January 29, 2004 4:54 PM
Subject: Re: logrotate and perl file handles. How do I know when to reopen my log file?


: TeamSolCO typed this on 01/29/2004 01:16 PM:
: > I have written a long running perl application to help me with some trivial administrative functions. Due to
the
: > volume of logging generated by this application, I am managing its output with logrotate. The problem I face is
that
: > perl is "following" the log files that logrotate swaps out. While this is probably intelligent on perl's part (to
: > follow the file descriptor rather than the file name), it is presenting a major problem for me. When "logfile"
becomes
: > "logfile.1" (etc), my application follows that move and "continues" logging to the new "logfile.1" instead of losing
: > that file handle -- a condition which I check for so that I can reopen the log file using the proper file name,
: > "logfile".
: > Ultimately, perl will only lose the file handle when logrotate finally deletes the "last file" in the log
rotation
: > scheme. Unfortunately, I will have lost a week's worth of logs because all that data will have been in the single
file.
: > Additionally, I'll have several empty log file copies named .1, .2, .3, etc. How can I set perl to NOT follow the
file
: > when it is renamed by logrotate, such that a condition will be created where I will be able to initialize the
: > replacement file?
: >
: > Thanks!!
: >
: > - William Kimball
: > "Programming is an art-form that fights back!"
: >
:
: Do you want to only monitor the original logfile or logfile.1 as well?
:
: You can use the perl module, File::Tail for monitoring logfile, even
: through logrotate.
:
: -Robby
:
:
: --
: #-------------------------------------------------------
: # Robby Russell, | Sr. Administrator / Lead Programmer
: # Command Prompt, Inc. | [url]http://www.commandprompt.com[/url]
: # [email]rrussell@commandprompt.com[/email] | Telephone: (503) 667.4564
: #-------------------------------------------------------
:
:

[John McKown]

What is happening is that once a file has been opened, the handle points
to the "inode" which is an internal "number". This "number" is unique for
a filesystem and identifies the file regardless of its "name" in the
directory. That is why your Perl program is "following" the "old name".
Anyway,

What you need to do is create an entry in /etc/logrotate.d. Look at one of
the entries in there for an example of how to code it. Oh, I think you've
already done this.

What your Perl program needs to do is:

1) write its PID to a known location. Many put this is /var/run. Something
like:

open (PIDFILE,">/var/run/myPerlScript.pid") or die "cannot open pid file
$!";
print PIDFILE "$$";
close(PIDFILE);
....

2) Determine the "signal" that you want logrotate to issue to tell your
script to reopen its log file. Many programs use the HUP signal. Create a
signal handler in your Perl program to field this signal to close and
reopen its log file. This is the main "magic" to get this working. I'd
make LOGHANDLE an "our" variable at the top of the Perl program. Replace
LOGHANDLE with the correct variable, of course <grin>.

$SIG{HUP} = sub {
close(LOGHANDLE);
open(LOGHANDLE,">/var/log/myPerlScript.log") or die "cannot open
logfile $!";
};

3) Create the aforementioned logrotate entry in /etc/logrotate.d something
like:

/var/log/myPerlScript.log {
notifempty
missingok
postrotate
/usr/bin/kill -HUP `cat /var/run/myPerlScript.pid

>/dev/null` 2>/dev/null || true

endscript
}

perhaps calling it "myPerlScript.log". Note - replace "myPerlScript" with
the actual name of your Perl script. Note that when the signal handler
actually executes, the previous myPerlScript.log file has already been
renamed, so the open in the signal handler create a new file with the old
name.


On Thu, 29 Jan 2004, TeamSolCO wrote:

> I have written a long running perl application to help me with some
> trivial administrative functions. Due to the
> volume of logging generated by this application, I am managing its
> output with logrotate. The problem I face is that
> perl is "following" the log files that logrotate swaps out. While this
> is probably intelligent on perl's part (to
> follow the file descriptor rather than the file name), it is presenting
> a major problem for me. When "logfile" becomes
> "logfile.1" (etc), my application follows that move and "continues"
> logging to the new "logfile.1" instead of losing
> that file handle -- a condition which I check for so that I can reopen
> the log file using the proper file name,
> "logfile".
> Ultimately, perl will only lose the file handle when logrotate
> finally deletes the "last file" in the log rotation
> scheme. Unfortunately, I will have lost a week's worth of logs
> because all that data will have been in the single file.
> Additionally, I'll have several empty log file copies named .1, .2, .3,
> etc. How can I set perl to NOT follow the file
> when it is renamed by logrotate, such that a condition will be created
> where I will be able to initialize the
> replacement file?
>
> Thanks!!
>
> - William Kimball
> "Programming is an art-form that fights back!"
>

--
Maranatha!
John McKown

[Teamsolco]

AWESOME! That's *EXACTLY* what I needed! I didn't realize I could use logrotate for tasks other than, well, just
rotating logs. Thanks!!

----- Original Message -----
From: "John McKown" <joarmc@swbell.net>
To: "Perl Beginners Mailing List" <beginners@perl.org>
Sent: Thursday, January 29, 2004 5:43 PM
Subject: Re: logrotate and perl file handles. How do I know when to reopen my log file?


: What is happening is that once a file has been opened, the handle points
: to the "inode" which is an internal "number". This "number" is unique for
: a filesystem and identifies the file regardless of its "name" in the
: directory. That is why your Perl program is "following" the "old name".
: Anyway,
:
: What you need to do is create an entry in /etc/logrotate.d. Look at one of
: the entries in there for an example of how to code it. Oh, I think you've
: already done this.
:
: What your Perl program needs to do is:
:
: 1) write its PID to a known location. Many put this is /var/run. Something
: like:
:
: open (PIDFILE,">/var/run/myPerlScript.pid") or die "cannot open pid file
: $!";
: print PIDFILE "$$";
: close(PIDFILE);
: ...
:
: 2) Determine the "signal" that you want logrotate to issue to tell your
: script to reopen its log file. Many programs use the HUP signal. Create a
: signal handler in your Perl program to field this signal to close and
: reopen its log file. This is the main "magic" to get this working. I'd
: make LOGHANDLE an "our" variable at the top of the Perl program. Replace
: LOGHANDLE with the correct variable, of course <grin>.
:
: $SIG{HUP} = sub {
: close(LOGHANDLE);
: open(LOGHANDLE,">/var/log/myPerlScript.log") or die "cannot open
: logfile $!";
: };
:
: 3) Create the aforementioned logrotate entry in /etc/logrotate.d something
: like:
:
: /var/log/myPerlScript.log {
: notifempty
: missingok
: postrotate
: /usr/bin/kill -HUP `cat /var/run/myPerlScript.pid
: >/dev/null` 2>/dev/null || true
: endscript
: }
:
: perhaps calling it "myPerlScript.log". Note - replace "myPerlScript" with
: the actual name of your Perl script. Note that when the signal handler
: actually executes, the previous myPerlScript.log file has already been
: renamed, so the open in the signal handler create a new file with the old
: name.
:
:
: On Thu, 29 Jan 2004, TeamSolCO wrote:
:
: > I have written a long running perl application to help me with some
: > trivial administrative functions. Due to the
: > volume of logging generated by this application, I am managing its
: > output with logrotate. The problem I face is that
: > perl is "following" the log files that logrotate swaps out. While this
: > is probably intelligent on perl's part (to
: > follow the file descriptor rather than the file name), it is presenting
: > a major problem for me. When "logfile" becomes
: > "logfile.1" (etc), my application follows that move and "continues"
: > logging to the new "logfile.1" instead of losing
: > that file handle -- a condition which I check for so that I can reopen
: > the log file using the proper file name,
: > "logfile".
: > Ultimately, perl will only lose the file handle when logrotate
: > finally deletes the "last file" in the log rotation
: > scheme. Unfortunately, I will have lost a week's worth of logs
: > because all that data will have been in the single file.
: > Additionally, I'll have several empty log file copies named .1, .2, .3,
: > etc. How can I set perl to NOT follow the file
: > when it is renamed by logrotate, such that a condition will be created
: > where I will be able to initialize the
: > replacement file?
: >
: > Thanks!!
: >
: > - William Kimball
: > "Programming is an art-form that fights back!"
: >
:
: --
: Maranatha!
: John McKown
:
:
: --
: To unsubscribe, e-mail: [email]beginners-unsubscribe@perl.org[/email]
: For additional commands, e-mail: [email]beginners-help@perl.org[/email]
: <http://learn.perl.org/> <http://learn.perl.org/first-response>
:
:
: