Professional Web Applications Themes

Apache mod_rewrite/RewriteMap prg: and perl script - Linux / Unix Administration

Sorry for the x-posting, but sometimes I just gotta do it! So many redundant-seeming newsgroups and all... Anyways, I have a server as follows: * RHL9 (OS version not important here, I don't think) * Apache 1.3.27 * PHP 4.3.2 * MySQL 3.23.56 I am using mod_rewrite to map username's last names to their row_id in my person database. The Perl script I am using to do this *works* I have tested it. It follows the KISS design philosphy. example I/O session with this script: # ./facmap.pl userlastname<ENTER> 53 It spits out the row_id if it gets a valid last ...

  1. #1

    Default Apache mod_rewrite/RewriteMap prg: and perl script

    Sorry for the x-posting, but sometimes I just gotta do it! So many
    redundant-seeming newsgroups and all...

    Anyways, I have a server as follows:
    * RHL9 (OS version not important here, I don't think)
    * Apache 1.3.27
    * PHP 4.3.2
    * MySQL 3.23.56


    I am using mod_rewrite to map username's last names to their row_id in my
    person database. The Perl script I am using to do this *works* I have
    tested it. It follows the KISS design philosphy.

    example I/O session with this script:
    # ./facmap.pl
    userlastname<ENTER>
    53

    It spits out the row_id if it gets a valid last name.

    Now I am using RewriteMap prg: to get this mapping in Apache as follows:

    RewriteMap faculty_map prg:/usr/local/bin/facmap.pl

    RewriteRule ^/faculty/([^/]*)(/*.*) /webdir/fp$2?id=${faculty_map:$1} [L]

    This *works*! I know it does. I have seen it! :)

    Now, the problem is that it *stops* *working*! After a certain amount of
    time, the map gets munged. I have to restart or reload or graceful apache
    to get the map working properly again!

    Things I do know:
    * The perl script works on its own.
    * The rewrite rules are doing waht I want them to do when I first start
    Apache
    * I upgraded the server software recently as follows:
    * Apache 1.3.23 -> 1.3.27
    * PHP 4.3.1 -> 4.3.2
    * MySQL 3.23.23 (?) -> 2.23.56
    * RHL 7.2 -> RHL9
    I mention these upgrades because I had the same basic rewrite rules
    before the upgrade and they never seemd to get munged.
    * I tried adding a RewriteLock directive but it did not seem to help.

    Things I do NOT know:
    * How much time passes before the problem occurs.
    * *what* is causing the problem
    * How to get the system to log this problem ( I tried bumping up the
    RewriteLogLevel to 2 and 3 but this did not seem to capture the event).


    Alright, thanks in advance!

    later...

    --
    Jeffrey D. Silverman | jeffrey AT jhu DOT edu
    Johns Hopkins University | Baltimore, MD
    Website | [url]http://www.wse.jhu.edu/newtnotes/[/url]

    Jeffrey Silverman Guest

  2. #2

    Default Re: Apache mod_rewrite/RewriteMap prg: and perl script

    "Jeffrey Silverman" <jeffreyjhu.edu> wrote in message news:<pan.2003.07.28.14.05.37.990760jhu.edu>...
    > Sorry for the x-posting, but sometimes I just gotta do it! So many
    > redundant-seeming newsgroups and all...
    comp.unix.admin is probably a little broad for this question, and
    alt.apache.configuration is a poorly-propogated unsupported newsgroup.
    I'd stick with comp.infosystems.[url]www.servers.unix[/url].

    > Now, the problem is that it *stops* *working*! After a certain amount of
    > time, the map gets munged. I have to restart or reload or graceful apache
    > to get the map working properly again!
    I can't solve your problem, but I know that I have seen some other reports
    of rewritemap programs getting out-of-order. I suggest you check the
    apache bug database.

    You noted that you checked the rewritelock, but make sure that it is working
    correctly. In particular, it probably needs to be in an httpd-writable
    directory.

    Joshua.
    Joshua Slive Guest

  3. #3

    Default Re: Apache mod_rewrite/RewriteMap prg: and perl script

    On Mon, 28 Jul 2003 11:23:22 -0700, Joshua Slive wrote:

    <snip!>
    > I can't solve your problem, but I know that I have seen some other reports
    > of rewritemap programs getting out-of-order. I suggest you check the
    > apache bug database.
    >
    > You noted that you checked the rewritelock, but make sure that it is
    > working correctly. In particular, it probably needs to be in an
    > httpd-writable directory.
    >
    > Joshua.
    I couldn't find *anything* in the Apache bug database!

    As far as I can tell, it does not seem to matter what the perms are on the
    directory where the lockfile is being written. I experimented as follows:

    "RewriteLock /usr/local/apache/lock/map.lock"

    chown root:nobody /usr/local/apache/lock/
    chmod 777 /usr/local/apache/lock/
    chown nobody:nobody /usr/local/apache/lock/
    chmod 777 /usr/local/apache/lock/
    chown root:root /usr/local/apache/lock/
    chmod 000 /usr/local/apache/lock/

    I even did (in httpd.conf):
    "RewriteLock /tmp/map.lock"

    i.e. move lockfile to world-writable /tmp.

    In all of the above cases and combinations, a lockfile was created.
    However, the lockfile was *ALWAYS* zero bytes! Is it supposed to be zero
    bytes?

    I'm stuck and get figure it out.

    Incidentally, I put an ugly hack in place to work around thihs problem:
    # crontab -e
    0 0-23/2 * * * /usr/local/apache/bin/apachectl graceful

    Ugh! brute-force ugliness.

    Thanks for your help...
    --
    Jeffrey D. Silverman | jeffrey AT jhu DOT edu
    Johns Hopkins University | Baltimore, MD
    Website | [url]http://www.wse.jhu.edu/newtnotes/[/url]

    Jeffrey Silverman Guest

  4. #4

    Default Re: Apache mod_rewrite/RewriteMap prg: and perl script

    > I am using mod_rewrite to map username's last names to their row_id in my
    > person database. The Perl script I am using to do this *works* I have
    > tested it. It follows the KISS design philosphy.
    >
    > example I/O session with this script:
    > # ./facmap.pl
    > userlastname<ENTER>
    > 53
    >
    > It spits out the row_id if it gets a valid last name.
    what does it do if you give it thousands of valid and invalid last
    names?
    and what happens if you start the script and leave it running whith no
    input for some hours and then go on (especially if the database is on
    a remote box - tcp connection might have gone dead for whatever
    reason)?
    remember this one script instance is running _all the time_ and if it
    blocks, so does your rewrite-map, so just testing once or twice is not
    really sufficient...
    > * I upgraded the server software recently as follows:
    > * Apache 1.3.23 -> 1.3.27
    > * PHP 4.3.1 -> 4.3.2
    > * MySQL 3.23.23 (?) -> 2.23.56
    > * RHL 7.2 -> RHL9
    > I mention these upgrades because I had the same basic rewrite rules
    > before the upgrade and they never seemd to get munged.
    > * I tried adding a RewriteLock directive but it did not seem to help.
    how is the database when the lockup occurs? but probably fine for a
    new connection...
    > Things I do NOT know:
    > * How much time passes before the problem occurs.
    can't this be determined from the logs? also it would be interesting
    to know the cirstances at lockup like high traffic, low or no
    traffic at all, always after n days of running or so etc pp.

    the two single problems i spent most time in debugging were:

    (1) a whole server farm keeling over at about the same time each
    workday (operators were placing bets on when it would happen) - some
    high paid consultant long gone had elected in his infinite wisdom to
    use a 16bit counter for requests in some library and when it
    overflowed the box went down the drain. we started looking for
    ressource leakage at first but when i had a closer look at the logs
    from our stress-test environment and saw exactly 2^16 requests until
    up we looked for counters and soon found this strike of genius...

    (2) a webserver talking to a backend just fine during the day and
    sporadically loosing contact at night - reason was some ssl connection
    timing out due to long disuse at night and needing ages to be rebuilt
    - under load the connection was never timing out and therefore all was
    fine. here too the occurence pattern was the key to look for ssl
    handshake problems.

    also you might post the script for some perl buffs here to look at and
    think twice about any other changes to the architecture except for the
    upgrade: database moved from webserver to separate system, maybe new
    firewall between web- and db-box and so on.

    joachim
    Joachim Ring Guest

  5. #5

    Default Re: Apache mod_rewrite/RewriteMap prg: and perl script

    On Mon, 28 Jul 2003 14:19:53 -0700, Joachim Ring wrote:
    >> I am using mod_rewrite to map username's last names to their row_id in my
    >> person database. The Perl script I am using to do this *works* I have
    >> tested it. It follows the KISS design philosphy.
    >>
    >> example I/O session with this script:
    >> # ./facmap.pl
    >> userlastname<ENTER>
    >> 53
    >>
    >> It spits out the row_id if it gets a valid last name.
    >
    > what does it do if you give it thousands of valid and invalid last
    > names?
    > and what happens if you start the script and leave it running whith no
    > input for some hours and then go on (especially if the database is on
    > a remote box - tcp connection might have gone dead for whatever
    > reason)?
    > remember this one script instance is running _all the time_ and if it
    > blocks, so does your rewrite-map, so just testing once or twice is not
    > really sufficient...
    >
    >> * I upgraded the server software recently as follows:
    >> * Apache 1.3.23 -> 1.3.27
    >> * PHP 4.3.1 -> 4.3.2
    >> * MySQL 3.23.23 (?) -> 2.23.56
    >> * RHL 7.2 -> RHL9
    >> I mention these upgrades because I had the same basic rewrite rules
    >> before the upgrade and they never seemd to get munged.
    >> * I tried adding a RewriteLock directive but it did not seem to help.
    >
    > how is the database when the lockup occurs? but probably fine for a
    > new connection...
    <snip!>

    Thank you for your reply!

    There *is* one bit of information I ommitted, I realize, and that is that
    "lockup" is not the symptom of this problem. That is, the system does not
    hang or crash.

    What *does* happen is that the script gets garbled. It spits out the wrong
    name->id map. It seems to change with each successive hit, too, that is,
    it does not stay garbled in a fixed, offset kind of manner. It is a
    different garble each time you hit it.

    I can't seem to tell if this is random or if it moves successively
    through each id in its list of id's.

    Also, I did not create the script with a continuous database connection in
    mind. The script looks up the mappings in the database *once*, creates an
    associative array based on this lookup, and uses that hash as its lookup
    table -- all in memory (presumably). (I always try to limit the number of
    times I actually need to access the database for *any* script or program I
    write, toot toot*) Thus there is little or no chance of
    db connection loss being the problem.

    thanks
    later...

    (* toot too == sound of me tooting my own horn)
    --
    Jeffrey D. Silverman | jeffrey AT jhu DOT edu
    Johns Hopkins University | Baltimore, MD
    Website | [url]http://www.wse.jhu.edu/newtnotes/[/url]

    Jeffrey Silverman Guest

  6. #6

    Default Re: Apache mod_rewrite/RewriteMap prg: and perl script

    "Jeffrey Silverman" <jeffreyjhu.edu> wrote in message news:<pan.2003.07.28.20.23.16.135560jhu.edu>...
    >
    > I couldn't find *anything* in the Apache bug database!
    You didn't look hard enough. I found at least:
    [url]http://nagoya.apache.org/bugzilla/show_bug.cgi?id=9534[/url]
    which has similar symptoms, but is for the 2.0 version.

    Probably not the same as your problem.

    There is also this one which may be related:
    [url]http://nagoya.apache.org/bugzilla/show_bug.cgi?id=14453[/url]

    Joshua
    Joshua Slive Guest

  7. #7

    Default Re: Apache mod_rewrite/RewriteMap prg: and perl script

    "Jeffrey Silverman" <jeffreyjhu.edu> wrote in message news:<pan.2003.07.29.17.30.22.351288jhu.edu>...
    > I tried using Apache's bugzilla. It was either not working or I wasn't
    > using it correctly. For example, typing in keywords into the search field
    > at this URL:
    >
    > [url]http://bugs.apache.org/[/url]
    >
    > returned results on *all* apache websites. i.e. it conducted a general
    > search on *.apache.org! This is not what I expected.
    That bug database is, as mentioned on the front page, obsolete.
    > Also, using the "new bug database", here:
    > [url]http://nagoya.apache.org/bugzilla/[/url]
    > (which is the root of the URLs you sent me, above) turned up nothing useful.
    >
    > That is the problem I often have with "Search" features -- if you don't
    > already know what the problem is or what too call it, you can't search on
    > keywords.
    >
    > I tried the most general keywords I coul -- "mod_rewrite" and "RewriteMap"
    > and camp up with nothing useful.
    mod_rewrite returns stuff for me.

    You should look at the instructions here:
    [url]http://httpd.apache.org/bug_report.html[/url]
    which tell you to start your query with the ALL keyword in order to get
    both open and closed reports.

    Joshua.
    Joshua Slive Guest

  8. #8

    Default Re: Apache mod_rewrite/RewriteMap prg: and perl script

    > Also, I did not create the script with a continuous database connection in
    > mind. The script looks up the mappings in the database *once*, creates an
    > associative array based on this lookup, and uses that hash as its lookup
    > table -- all in memory (presumably).
    hm, then i don't really understand the need for an external rewrite
    map in the first place. the only reason i could think of to use one
    would be the need for online access to the user-database. without
    that, a flat file rewrite-map will serve you better.
    > (I always try to limit the number of
    > times I actually need to access the database for *any* script or program I
    > write, toot toot*)
    which is a good idea.
    > Thus there is little or no chance of
    > db connection loss being the problem.
    ok, but anyways you might want to configure a regular flat-file
    rewrite-map and update that by running a slightly modified version of
    your script from cron. that way you even get updates at regular
    intervals unlike your version which won't update the map unlike apache
    (and therefore the external rewrite map) is restarted.

    joachim
    Joachim Ring Guest

  9. #9

    Default Re: Apache mod_rewrite/RewriteMap prg: and perl script

    On Mon, 04 Aug 2003 11:17:44 -0700, Joachim Ring wrote:

    <snip!>
    > ok, but anyways you might want to configure a regular flat-file
    > rewrite-map and update that by running a slightly modified version of your
    > script from cron. that way you even get updates at regular intervals
    > unlike your version which won't update the map unlike apache (and
    > therefore the external rewrite map) is restarted.
    >
    > joachim
    Great! I never would have thought of that, thanks for the idea!
    --
    Jeffrey D. Silverman | jeffrey AT jhu DOT edu
    Johns Hopkins University | Baltimore, MD
    Website | [url]http://www.wse.jhu.edu/newtnotes/[/url]

    Jeffrey Silverman Guest

Similar Threads

  1. Replies: 4
    Last Post: October 9th, 08:21 PM
  2. Apache returns 200 OK, but my perl script sends 302 REDIRECT
    By Alan J. Flavell in forum PERL Miscellaneous
    Replies: 12
    Last Post: August 23rd, 06:29 AM
  3. Catching Apache error with a Perl script
    By PapaBear in forum PERL Miscellaneous
    Replies: 2
    Last Post: August 20th, 12:56 PM
  4. Replies: 0
    Last Post: July 29th, 11:44 AM
  5. Replies: 3
    Last Post: July 16th, 01:06 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139