Professional Web Applications Themes

silent crash - MySQL

Hi, Since a few months, we have a strange issue with mysql server ( last release for win, on a xp SP2 box ). Randomly, Mysql daemon stops responding. ( estimated frequency : about 1 time per week or per 2 week ) I mean : * No way to connect ( by tcp/ip or by running the mysql command line tool ). * Stop and then restart the Mysql service is unsuccessful. The only "workaround" is to ... reboot the computer ! And then, all's ok for a random duration. When I read the Mysql log, no error is ...

  1. #1

    Default silent crash


    Hi,

    Since a few months, we have a strange issue with mysql server ( last
    release for win, on a xp SP2 box ).

    Randomly, Mysql daemon stops responding.
    ( estimated frequency : about 1 time per week or per 2 week )

    I mean :
    * No way to connect ( by tcp/ip or by running the mysql command line tool ).
    * Stop and then restart the Mysql service is unsuccessful.

    The only "workaround" is to ... reboot the computer !
    And then, all's ok for a random duration.

    When I read the Mysql log, no error is written.
    I never saw this kind of issue, and so :
    - I updated Mysql server version.
    - I replaced the computer which run Mysql for a new one with more memory
    and cpu ( 512Mo / 1,8Ghz ).

    So, I'm tired of this issue ... does any one have an idea, something to
    check ??

    Thanks you everybody for any help.

    Luc.










    Luc Guest

  2. #2

    Default Re: silent crash

    Luc wrote: 

    What about anti-virus programs and such, tried to turn them off?
     

    Do you dare to update to the Unix world?
    It's considered more stable.


    --

    //Aho
    J.O. Guest

  3. #3

    Default Re: silent crash

    I would look at windows application log files as well. plus i believe you can
    turn on performance monitoring so that you know when and/or how it starts to overload.
    lark Guest

  4. #4

    Default Re: silent crash

    are you sure it's independant of a long running query or loop?

    --

    Jack Vamvas
    ___________________________________
    The latest IT jobs - www.ITjobfeed.com
    <a href="http://www.itjobfeed.com">UK IT Jobs</a>


    "Luc" <com> wrote in message
    news:et96ro$fgk$tiscali.fr... 


    Jack Guest

  5. #5

    Default Re: silent crash

    Jack Vamvas a écrit : 
    I think so ... but I have no way to be 100% sure.

    As all scripts started on this computer are Perl scripts, and so I tried
    to kill all perl processes when issue appears. But it didn't solved the
    issue.

    All these scripts asking the mysql server were written by myself ...
    it's not an insurance of "good work" but I know that no one else run
    queries to this mysql server.







    Luc Guest

  6. #6

    Default Re: silent crash

    J.O. Aho a écrit : 

    I have desactivated it ( Mc Afee viruscan ).
    But, to do a test, I uninstalled it fully since a few days.

    I'll tell you if it seems to solves the issue ...
     

    I come from the unix world ... but my company is running on the
    Microsoft world and always with many MsSQL servers ! :o(
    So It was hard to have the agreement of my boss to use MySQL ( because
    dba's knows only MsSQL, here ), and to use Perl instead of vb/asp.
    My company think about migrating to the unix world, but I can't choose
    now to install a box with another OS. It's another problem :o(

    Anyway, in my previous company, I ran successfully a MySQL 4.1 server
    for years with no crash problems ...
    And the queries/load average were bigger.
    Luc Guest

  7. #7

    Default Re: silent crash

    Luc wrote: 
    >
    > I have desactivated it ( Mc Afee viruscan ).
    > But, to do a test, I uninstalled it fully since a few days.
    >
    > I'll tell you if it seems to solves the issue ...[/ref]

    If it turns out that the virus scanner is the problem, maybe that could be an
    argument to try another environment. I wish you the best of luck.

    --

    //Aho
    J.O. Guest

  8. #8

    Default Re: silent crash

    Luc a écrit : 
    >
    > I have desactivated it ( Mc Afee viruscan ).
    > But, to do a test, I uninstalled it fully since a few days.
    >
    > I'll tell you if it seems to solves the issue ...[/ref]

    Unfortunately, I had a new "crash" of MySQL today.

    I'm despaired :'(




    Luc Guest

  9. #9

    Default Re: silent crash

    Luc wrote: 
    >>
    >> I have desactivated it ( Mc Afee viruscan ).
    >> But, to do a test, I uninstalled it fully since a few days.
    >>
    >> I'll tell you if it seems to solves the issue ...[/ref]
    >
    > Unfortunately, I had a new "crash" of MySQL today.
    >
    > I'm despaired :'([/ref]

    If you would call some support place they would most likely recommend
    reinstall microsoft, but before that, is it possible for your to add logging
    in your scripts, time and task? This way you could see if the crash happens at
    the same place each time or if it's all random.

    Running the mysql with strace would most likely be the best, but I doubt there
    are such tool for microsoft.

    --

    //Aho
    J.O. Guest

  10. #10

    Default Re: silent crash

    J.O. Aho a écrit :
     

    I already have some logging functions in my scripts.
    But currently, the only logged errors are about a connect failure.
    And before that connect failure, all scripts seems to return normally
    data ( all queries are successful ).

    Sometimes, issue appears when I run a script which do a basic select (
    where clause is on a specific id only ) on a single table and with only
    five hundred rows at all ... I think it's not to much, don't you ?

    And Apache logs aren't very interesting too.

    The strangest thing is that stopping/restarting the mysql service don't
    solve issue ... I don't understand why MySQL is able to restart with no
    errors, and why it continue to "deny/refuse" connections after. And so,
    why a "reboot" solve the issue.
     

    I don't know if I can use strace, but I'll try to do so.

    Anyway, thanks you very much for your answers ( I'll continue to tell
    you the results of the tests/logs if I find something interesting ).
    Luc Guest

  11. #11

    Default Re: silent crash

    Luc wrote: 
    >
    > I already have some logging functions in my scripts.
    > But currently, the only logged errors are about a connect failure.
    > And before that connect failure, all scripts seems to return normally
    > data ( all queries are successful ).
    >
    > Sometimes, issue appears when I run a script which do a basic select (
    > where clause is on a specific id only ) on a single table and with only
    > five hundred rows at all ... I think it's not to much, don't you ?
    >
    > And Apache logs aren't very interesting too.
    >
    > The strangest thing is that stopping/restarting the mysql service don't
    > solve issue ... I don't understand why MySQL is able to restart with no
    > errors, and why it continue to "deny/refuse" connections after. And so,
    > why a "reboot" solve the issue.[/ref]

    One thing that came in mind is that there would be a crash in a mysql
    library (dll) and microsft can't flush it out at a restart.

    To confirm this I guess you would need to wait to next crash, see if
    there is some mysql related task left "alive" when you have terminated
    the mysql (it could be that task that can't be killed and prevents that
    mysql will start properly again).
    Or you could try to upgrade the mysql (if you are allowed to do that)
    and see if the trouble disappears.


    --

    //Aho
    J.O. Guest

  12. #12

    Default Re: silent crash

    Luc <com> wrote: 

    Is the mysqld-nt.exe process running? Are there messages in the MySQL
    error log? The error log is $INSTALLDIR/Data/$HOSTNAME.err

    If mysqld is running, but you cannot connect, it's probably not a
    MySQL problem. What hostname do you use to connect to MySQL?
    localhost, 127.0.0.1, $REAL_IP_ADDR? Tried all of them? What exactly
    happens when you try to connect? Error message, hanging program,
    program returning without a message?
     

    Stopping mysqld via MMC does not work for me either. What always works:
    in command.exe, run: mysqladmin -u root shutdown
    If that does not work, just end the mysqld process (this may damage
    your data). Then start MySQL via MMC again.
     

    What log? Is there anything in there? Show your my.ini!

    Disable Firewall (or put in yze mode to have it asking you for
    every unknown action; MySQL is TCP/3306). COnfigure your virus
    scanner to not check files in the MySQL dir.

    Get the sysinternals tools. Process Explorer, Process Monitor and File
    Monitor will show you what mysqld does and if it interferes (i.e. by
    file locking) with other processes.


    XL
    --
    Axel Schwenke, Support Engineer, MySQL AB

    Online User Manual: http://dev.mysql.com/doc/refman/5.0/en/
    MySQL User Forums: http://forums.mysql.com/
    Axel Guest

  13. #13

    Default Re: silent crash

    Axel Schwenke a écrit : 

    Yes, mysqld-nt is always running.
    Issue appeared 2 times today, but nothing in the .err.

    Something was different today :
    First time, I was unable to connect ( for instance, via mysql admin ).

    Second time, I was already running mysqlAdmin, when a script "started"
    to hang. MysqlAdmin didn't crash ( previously, MysqlAdmin hang-up too ).

    So, I was able to see that user bdc ( it's the one which is used to run
    queries via scripts ) was connected and that status was "sleep" ( while
    the status for "root" was "query" ).
    Status for queries of bdc never changed.
    I killed it and tried again : Always the same.

     

    When issue appears, I tried the FQDN, the ip adress, localhost and
    127.0.0.1.
     

    I'm so sorry : I've read this message now, and issue appeared earlier in
    the day.
    So, I didn't make this test.

    But, when I stop Mysql via MMC, an info message is created :
    <<
    C:\Program Files\MySQL\MySQL Server 5.0\bin\mysqld-nt: Shutdown complete

    For more information, see Help and Support Center at http://www.mysql.com. [/ref]
    And then, when I start it again :
    <<
    C:\Program Files\MySQL\MySQL Server 5.0\bin\mysqld-nt: ready for
    connections.
    Version: '5.0.27-community-nt' socket: '' port: 3306 MySQL Community
    Edition (GPL)

    For more information, see Help and Support Center at http://www.mysql.com. [/ref]
     

    Ok, I killed the process, restarted it via MMC but it was always the same.
     

    Would you like the full file or only a few params/values ?
     

    It's very unsafe, but Firewall and antivirus are uninstalled ... to test :o/
    (it's not a good solution, but computer is in a dmz inside an internal
    network ... ).

     

    So, I'll install these tools tomorrow to try to give you more details at
    next crash.

    Thanks you very much for your help !

    Luc Guest

  14. #14

    Default Re: silent crash

    J.O. Aho a écrit :
     

    I printed out the list of tasks, stopped mysql when issue appeared, then
    tried to find a mysql process always running.
    But I found nothing.

    Thanks you, again, for your answers ;o)
    Luc Guest

  15. #15

    Default Re: silent crash

    Axel Schwenke a écrit : 

    Today, I had the issue 2 times again.

    Via Process Explorer, and Tcp View, I killed all mysql tasks.
    But when restarted, issue was always here.
    So I rebooted the unit again.
     

    You'll found it (uncommented) at the end of my post.
     

    The "more" interesting one seems to be Process Monitor.
    But I wasn't able to see something which appears only when mysql stop
    answering.

    I saw many "buffer overflow" on both csrss.exe and perl.exe,
    but nothing regarding mysql ... :o(
    Ex:
    13:38:30 csrss.exe:624 QUERY INFORMATION
    C:\WINDOWS\WinSxS\Manifests\x86_Microsoft.Windows. Common-Controls_6595b64144ccf1df_6.0.2600.1891_x-ww_7d3bbc01.Manifest
    BUFFER OVERFLOW FileFsVolumeInformation

    When issue appears, tcpview let's me see a few ( 3-4 ) Mysql processes
    in this state ( it never end, end ending manually these processes don't
    solve issue )

    TCP view
    mysqld-nt.exe:1660 TCP helpdesk.ourcompany.com:3306
    helpdesk.ourcompany.com:1200 FIN_WAIT2


    I don't know what is the "FIN_WAIT2" state, and if it's normal.


    Here is the my.ini file :
    ____________________________________

    [client]

    port=3306

    [mysql]

    default-character-set=latin1

    [mysqld]

    port=3306
    basedir="C:/Program Files/MySQL/MySQL Server 5.0/"
    datadir="C:/Program Files/MySQL/MySQL Server 5.0/Data/"
    default-character-set=latin1
    default-storage-engine=INNODB
    max_connections=100
    query_cache_size=23M
    table_cache=256
    tmp_table_size=26M
    thread_cache_size=8
    myisam_max_sort_file_size=100G
    myisam_max_extra_sort_file_size=100G
    myisam_sort_buffer_size=51M
    key_buffer_size=64M
    read_buffer_size=64K
    read_rnd_buffer_size=256K
    sort_buffer_size=256K
    innodb_data_home_dir="D:/MySQL Datafiles/"
    innodb_additional_mem_pool_size=2M
    innodb_flush_log_at_trx_commit=1
    innodb_log_buffer_size=1M
    innodb_buffer_pool_size=68M
    innodb_log_file_size=14M
    innodb_thread_concurrency=8
    enable-pstack
    core-file



    Luc Guest

  16. #16

    Default Re: silent crash

    Last information :

    Shortly after killing mysql processes, I tried to connect via
    mysqlAdministrator.

    In TcpView, I see some exchanges, and then the following message appears
    ( on mysqladminstrator ):
    mysql error 2013 : lost connection during query

    .... and then, all mysql processes are in "fin_wait2" state.


    Thanks you very much for any ideas.

    Luc
    Luc Guest

  17. #17

    Default Re: silent crash

    Luc wrote:
     

    Two descriptions found with google:

    "FIN_WAIT2 is a state when the server closes the socket on its side but
    does not receive an acknowledgement of the close from the client.

    Connection is closed, and the socket is waiting for a shutdown from the remote
    end.


    --

    //Aho
    J.O. Guest

  18. #18

    Default Re: silent crash


    Since a few days, issue appeared at least one time / day.

    I made the following tests :

    - Try to connect to the Mysql server from the computer running it, via
    Mysql Administrator or mysql command line : It don't works, and the
    following error appears when using the mysql command line tool :
    <<
    mysql --host=helpdesk --port=3333 --user=root --password=*****
    ERROR 2013 (HY000): Lost connection to MySQL server at 'reading
    authorization packet', system error: 2 [/ref]

    - And then, I tried to connect to the MySQL server from another computer:
    * Via Mysql Administrator : unsuccessfull.
    * Via mysql command line tool : successfull !!!
    But ( sometimes ), when I try to do a basic select on a table ( not
    always the same table ), connection is lost.


    I don't know if these tests can be useful ... as issue become very
    "obstructing" for our work, I have only two choices :
    - find a way to solve this "silent crash" issue shortly.
    - Use a working Mssql server instead :o((

    Anyway, thanks you all for your previous answers/help.







    Luc Guest

  19. #19

    Default Re: silent crash

    Luc wrote: [/ref][/ref]

    Could it be that the hard drive is worn out and you get read/write errors that
    causes this? This could explain why you sometimes have managed to log to the
    sql server.

    Did you check the mysql bugzilla for this problem? A fast query on google.com
    gave a few bugzilla listings for similar error messages.



    --

    //Aho
    J.O. Guest

Similar Threads

  1. Replies: 3
    Last Post: August 11th, 10:45 PM
  2. Silent installation not very silent
    By M4g3 in forum Macromedia Shockwave
    Replies: 2
    Last Post: January 8th, 11:08 PM
  3. Silent uninstall not all that silent
    By buckston in forum Macromedia Flash Player
    Replies: 2
    Last Post: March 26th, 12:07 AM
  4. InDesign CS CE non-stop CRASH, CRASH, CRASH, CRASH, CRASH, CRASH
    By kajzica@adobeforums.com in forum Adobe Indesign Windows
    Replies: 15
    Last Post: July 2nd, 07:57 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139