Professional Web Applications Themes

Threads troubles - PERL Miscellaneous

Hi, I am trying to write a multiuser server (for games, chat, etc) in Perl. In time, this program might have to service >500 users simultaneously, who would be connected for some time. So I thought about using threads instead of forking a new process for every user, in order to keep my process table healthy and minimize inter-process communication overhead. I wrote simple server and client scripts to test the theory, but it does not work at all like I expected. The scripts are short enough, and I have cut away any unnecessary parts, so I included them both ...

  1. #1

    Default Threads troubles

    Hi,

    I am trying to write a multiuser server (for games, chat, etc) in Perl.
    In time, this program might have to service >500 users simultaneously,
    who would be connected for some time. So I thought about using threads
    instead of forking a new process for every user, in order to keep my
    process table healthy and minimize inter-process communication overhead.

    I wrote simple server and client scripts to test the theory, but it
    does not work at all like I expected. The scripts are short enough,
    and I have cut away any unnecessary parts, so I included them both at
    the end of this posting.

    First problem - threads seem to be working fine, but when I look at the
    output of ps, there are still seperate processes for each connection.
    Shouldn't the threads all be contained in one large process? I'm using
    an ithreads-enabled current version of Perl with Linux 2.4.19.

    Second problem - the process (group) just grows and grows with every
    connection, and no memory seems to be released after a client
    disconnects. I am aware that perl usually does not return freed memory
    to the system, but I was hoping that the memory would at least be
    reused for the next connection. This is what it looks like:

    PID %CPU %MEM VSZ RSS TTY COMMAND

    4176 1.3 1.2 4924 3168 pts/6 | | \_ perl svr_thr_x.pl
    (server started, no connections yet)

    4176 1.0 1.7 8288 4436 pts/6 | | \_ perl svr_thr_x.pl
    4182 0.0 1.7 8288 4436 pts/6 | | \_ perl svr_thr_x.pl
    4183 0.0 1.7 8288 4436 pts/6 | | \_ perl svr_thr_x.pl
    (first client)

    4176 1.0 2.0 11384 5356 pts/6 | | \_ perl svr_thr_x.pl
    4182 0.0 2.0 11384 5356 pts/6 | | \_ perl svr_thr_x.pl
    4183 0.0 2.0 11384 5356 pts/6 | | \_ perl svr_thr_x.pl
    4187 0.0 2.0 11384 5356 pts/6 | | \_ perl svr_thr_x.pl
    (first client + second client)

    4176 0.6 2.0 11368 5348 pts/6 | | \_ perl svr_thr_x.pl
    4182 0.0 2.0 11368 5348 pts/6 | | \_ perl svr_thr_x.pl
    (both clients disconnected)

    4176 0.5 2.4 14440 6244 pts/6 | | \_ perl svr_thr_x.pl
    4182 0.0 2.4 14440 6244 pts/6 | | \_ perl svr_thr_x.pl
    4193 0.0 2.4 14440 6244 pts/6 | | \_ perl svr_thr_x.pl
    (third client)


    Third problem - I have absolutely no idea why, but when I run the
    client script in another terminal (or on another host even), and hit
    Ctrl-C while the client is still receiving data from the server, the
    server(!) will quit (and the client too, of course).


    Can anybody explain to me what is happening? Is there a better approach
    than using threads? Should I just fork away for each new client?


    Thanks a lot in advance,
    stefan



    ===[ file svr_thr_x.pl ]==================================================

    #!/usr/bin/perl

    use strict;
    use warnings;

    use threads qw(yield);
    use threads::shared;

    use Socket;
    use POSIX qw(strftime);
    use IO::Handle;

    use constant EOL => "\015\012";

    # configuration
    my $SVR_NAME = "X v0.001"; # name string
    my $SVR_ADDR = undef; # set to undef for INADDR_ANY
    my $SVR_PORT = 4887; # default port is 4887
    my $MAXCLIENTS = 100; # SOMAXCONN for system max

    # listen on SVR_ADDR:SVR_PORT
    $SVR_ADDR = $SVR_ADDR ? inet_aton($SVR_ADDR) : INADDR_ANY;
    socket(SERVER, PF_INET, SOCK_STREAM, getprotobyname("tcp"))
    or die "Error opening socket: $!\n";
    setsockopt(SERVER, SOL_SOCKET, SO_REUSEADDR, pack("l", 1))
    or die "Error trying to set socket reusable option: $!\n";
    bind(SERVER, pack_sockaddr_in($SVR_PORT, $SVR_ADDR))
    or die "Error on bind: $!\n";
    listen(SERVER, $MAXCLIENTS)
    or die "Error on listen: $!\n";

    print "$SVR_NAME server running on port $SVR_PORT\n";

    # main loop
    my $curr_client_id = 0;
    for (;;) {
    my $cl_id = sprintf "CLIENT_%010d", $curr_client_id++;
    my $cl_sock = accept(my $client, SERVER);
    threads->create("serve_client", $client, $cl_id, $cl_sock);
    }

    # spawned threads use this sub
    sub serve_client {
    my ($client, $cl_id, $cl_sock) = _;
    my ($cl_port, $cl_addr) = unpack_sockaddr_in($cl_sock);
    my $cl_name = gethostbyaddr($cl_addr, AF_INET);
    print "connection from $cl_name:$cl_port [", inet_ntoa($cl_addr), "]\n";
    autoflush $client 1;
    for (1 .. 10) {
    print $client "... talking to $cl_name [id $cl_id] ...", EOL;
    sleep 1;
    }
    close $client;
    threads->self->join;
    }


    ===[ file client_x.pl ]================================================== =

    #!/usr/bin/perl

    use strict;
    use warnings;

    use Socket;

    my $host = "localhost";
    my $port = 4887;
    my $svr_sock = sockaddr_in($port, inet_aton($host));

    socket(SERVER, PF_INET, SOCK_STREAM, getprotobyname("tcp"));
    connect(SERVER, $svr_sock) or die "Error - could not connect: $!";
    while (defined(my $line = <SERVER>)) {
    print $line;
    }
    close (SERVER);


    Stefan Weiss Guest

  2. #2

    Default Re: Threads troubles

    On Tue, 09 Sep 2003 03:19:46 GMT,
    Stefan Weiss <spaceman-a299c-20030925ausgehaucht.sensenmann.at> wrote:
    > Hi,
    >
    > I am trying to write a multiuser server (for games, chat, etc) in Perl.
    > In time, this program might have to service >500 users simultaneously,
    > who would be connected for some time. So I thought about using threads
    > instead of forking a new process for every user, in order to keep my
    > process table healthy and minimize inter-process communication overhead.
    [snip]
    >
    > First problem - threads seem to be working fine, but when I look at the
    > output of ps, there are still seperate processes for each connection.
    > Shouldn't the threads all be contained in one large process? I'm using
    > an ithreads-enabled current version of Perl with Linux 2.4.19.
    Threads can show up as different processes. Check you ps doentation or
    you kernel doentation for details on how to identify threads (and
    I guess the perl docs too?)

    [snip second problem of memory usage which I can't help with]
    > Third problem - I have absolutely no idea why, but when I run the
    > client script in another terminal (or on another host even), and hit
    > Ctrl-C while the client is still receiving data from the server, the
    > server(!) will quit (and the client too, of course).
    That's probably something along of the lines of a SIGPIPE terminating
    your process. Catch the signal you are getting when writing to the broken
    socket, clean up, and continue.

    > Can anybody explain to me what is happening? Is there a better approach
    > than using threads? Should I just fork away for each new client?
    I know very little about perl threads so I'm not going to try and comment
    on the threading stuff.

    A third solution, to forking and threading, is to do the multiplexing
    yourself using something like select (perldoc -f select). Doing
    that can often be simpler for things like chat servers where the
    connections are not independant (data read from one will be sent to
    others, common data will be updated, etc) since all the data is
    shared and there are no race conditions or locking issues (since there's
    only one 'thread' executing).

    --
    Sam Holden

    Sam Holden Guest

  3. #3

    Default Re: Threads troubles

    On Tue, 09 Sep 2003 03:19:46 GMT, Stefan Weiss wrote:
    > Hi,
    >
    > I am trying to write a multiuser server (for games, chat, etc) in Perl.
    > In time, this program might have to service >500 users simultaneously,
    > who would be connected for some time. So I thought about using threads
    > instead of forking a new process for every user, in order to keep my
    > process table healthy and minimize inter-process communication overhead.
    >
    > I wrote simple server and client scripts to test the theory, but it
    > does not work at all like I expected. The scripts are short enough,
    > and I have cut away any unnecessary parts, so I included them both at
    > the end of this posting.
    >
    > First problem - threads seem to be working fine, but when I look at the
    > output of ps, there are still seperate processes for each connection.
    > Shouldn't the threads all be contained in one large process? I'm using
    > an ithreads-enabled current version of Perl with Linux 2.4.19.
    Surprise! Linux's "threads" are lightweight processes that share memory
    rather than copy-on-write it.
    > Second problem - the process (group) just grows and grows with every
    > connection, and no memory seems to be released after a client
    > disconnects. I am aware that perl usually does not return freed memory
    > to the system, but I was hoping that the memory would at least be
    > reused for the next connection. This is what it looks like:
    Which version of Perl are you using? I have it on good authority that
    version 5.8.0 has a number of leaks that have since been fixed.

    Then again, perhaps it's not good to join a thread from itself. I'm not
    familiar enough with the intricacies of iThreads to say for sure.
    > Third problem - I have absolutely no idea why, but when I run the
    > client script in another terminal (or on another host even), and hit
    > Ctrl-C while the client is still receiving data from the server, the
    > server(!) will quit (and the client too, of course).
    You should probably trace the source of the exit. The easiest way is to
    throw in debugging warn or print statements to see what the server's
    doing just before it stops.
    > Can anybody explain to me what is happening? Is there a better approach
    > than using threads? Should I just fork away for each new client?
    Perl's threads use about half a megabyte apiece, and that's usually on
    the low side of the estimate. 300 of them will require a minimum of
    150MB of memory. That figure will rise quickly if you start building
    objects and using modules.

    You may want to consider a single-process, single-thread design, based
    on four-argument select(), IO::Select, or a higher-level networking
    library like POE.
    >===[ file svr_thr_x.pl ]==================================================
    >
    > #!/usr/bin/perl
    >
    > use strict;
    > use warnings;
    >
    > use threads qw(yield);
    > use threads::shared;
    >
    > use Socket;
    > use POSIX qw(strftime);
    > use IO::Handle;
    >
    > use constant EOL => "\015\012";
    >
    > # configuration
    > my $SVR_NAME = "X v0.001"; # name string
    > my $SVR_ADDR = undef; # set to undef for INADDR_ANY
    > my $SVR_PORT = 4887; # default port is 4887
    > my $MAXCLIENTS = 100; # SOMAXCONN for system max
    >
    > # listen on SVR_ADDR:SVR_PORT
    Consider IO::Socket::INET here.
    > $SVR_ADDR = $SVR_ADDR ? inet_aton($SVR_ADDR) : INADDR_ANY;
    > socket(SERVER, PF_INET, SOCK_STREAM, getprotobyname("tcp"))
    > or die "Error opening socket: $!\n";
    > setsockopt(SERVER, SOL_SOCKET, SO_REUSEADDR, pack("l", 1))
    > or die "Error trying to set socket reusable option: $!\n";
    > bind(SERVER, pack_sockaddr_in($SVR_PORT, $SVR_ADDR))
    > or die "Error on bind: $!\n";
    > listen(SERVER, $MAXCLIENTS)
    > or die "Error on listen: $!\n";
    >
    > print "$SVR_NAME server running on port $SVR_PORT\n";
    >
    > # main loop
    > my $curr_client_id = 0;
    > for (;;) {
    > my $cl_id = sprintf "CLIENT_%010d", $curr_client_id++;
    > my $cl_sock = accept(my $client, SERVER);
    > threads->create("serve_client", $client, $cl_id, $cl_sock);
    > }
    >
    > # spawned threads use this sub
    > sub serve_client {
    > my ($client, $cl_id, $cl_sock) = _;
    > my ($cl_port, $cl_addr) = unpack_sockaddr_in($cl_sock);
    > my $cl_name = gethostbyaddr($cl_addr, AF_INET);
    > print "connection from $cl_name:$cl_port [", inet_ntoa($cl_addr), "]\n";
    > autoflush $client 1;
    > for (1 .. 10) {
    You don't test whether print() succeeds here. How do you know when the
    client has disconnected?
    > print $client "... talking to $cl_name [id $cl_id] ...", EOL;
    > sleep 1;
    > }
    > close $client;
    Is it okay for a thread to join itself? This seems circular; would it
    be the cause of memory not being released?
    > threads->self->join;
    > }
    --
    Rocco Caputo - [email]rcaputopobox.com[/email] - [url]http://poe.perl.org/[/url]
    Rocco Caputo Guest

  4. #4

    Default Re: Threads troubles

    Hi,

    many thanks to you and Sam Holden for your fast and very helpful
    answers.

    Rocco Caputo <trocpobox.com> wrote:
    > Surprise! Linux's "threads" are lightweight processes that share memory
    > rather than copy-on-write it.
    That explains a lot. I will take Sam's advice and familiarize myself
    with the way Linux handles threads before I start using them again.
    > Which version of Perl are you using? I have it on good authority that
    > version 5.8.0 has a number of leaks that have since been fixed.
    Spot on. Perl 5.8.0 running here - I thought that was a reasonably
    current version but I guess I'm a bit out of touch.
    > Then again, perhaps it's not good to join a thread from itself. I'm not
    > familiar enough with the intricacies of iThreads to say for sure.
    I'm not saying that I am familiar with iThreads, but calling join from
    a thread does not seem to be problematic. I have found this to be the
    only way to effectively close the client socket (filehandle), and I
    have seen people-who-know-what-they-are-doing detach()ing threads
    from inside. But then again, I am just starting to understand Perl
    threads.
    > You may want to consider a single-process, single-thread design, based
    > on four-argument select(), IO::Select, or a higher-level networking
    > library like POE.
    I am currently downloading POE, and it looks very promising. The reason
    why I did not go for a high-level framework in the first place is that
    I wanted to understand what exactly was happening, before I use
    abstractions and high-level interfaces to hide the gory details :)
    >>===[ file svr_thr_x.pl ]==================================================
    [..]
    > Consider IO::Socket::INET here.
    Same reason - understand the low level first, then use a specialized
    module.
    >> print "connection from $cl_name:$cl_port [", inet_ntoa($cl_addr),
    >> "]\n"; autoflush $client 1;
    >> for (1 .. 10) {
    >
    > You don't test whether print() succeeds here. How do you know when the
    > client has disconnected?
    Point taken.

    Thanks again for your advice. I will try multiplexing with select()
    now, and then probably use POE.


    cheers,
    stefan

    Stefan Weiss Guest

Similar Threads

  1. Web Service and Threads
    By wbarr@tin-man.co.uk in forum ASP.NET Web Services
    Replies: 1
    Last Post: June 15th, 01:28 PM
  2. threads in perl
    By Igor Ryaboy in forum PERL Beginners
    Replies: 24
    Last Post: March 3rd, 06:59 AM
  3. webrick, threads, and i/o
    By Ara.T.Howard in forum Ruby
    Replies: 4
    Last Post: October 4th, 03:11 PM
  4. Objects, threads and so on
    By Lists Perl Org in forum PERL Beginners
    Replies: 6
    Last Post: October 2nd, 06:20 AM
  5. Sockets and threads
    By Mark J. Reed in forum Ruby
    Replies: 1
    Last Post: August 26th, 07:38 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139