Professional Web Applications Themes

When threads block - Ruby

It's difficult to do any serious multi-threaded network programming when the whole process blocks on system calls like read, write, and select. Is this on a TODO list somewhere, or should I just accept it as a fact of life? (and that may mean using a different language.)...

  1. #1

    Default When threads block

    It's difficult to do any serious multi-threaded network programming when
    the whole process blocks on system calls like read, write, and select. Is
    this on a TODO list somewhere, or should I just accept it as a fact of
    life? (and that may mean using a different language.)
    Hans Fugal Guest

  2. #2

    Default Re: When threads block

    Hi,

    At Wed, 24 Sep 2003 10:16:53 +0900,
    Hans Fugal wrote:
    > It's difficult to do any serious multi-threaded network programming when
    > the whole process blocks on system calls like read, write, and select. Is
    > this on a TODO list somewhere, or should I just accept it as a fact of
    > life? (and that may mean using a different language.)
    What's your platform?

    If you use Windows, it's a known problem but hard to fix with
    current thread implementation.

    --
    Nobu Nakada

    nobu.nokada@softhome.net Guest

  3. #3

    Default Re: When threads block

    > If you use Windows, it's a known problem but hard to fix with
    > current thread implementation.
    When will we get a new thread implementation, then? :-)

    Seriously, could you outline how these things are implemented on other
    platforms to avoid blocking the entire process? Why can't this trick be used
    on the windows platform? I'd like to help if I can in any way.

    Another question - is true thread safety on the TODO list for ruby? Oh and
    this begs the third question: Is there a release plan for ruby? I would love
    to know what kind of goodies I can expect for future releases?

    Cheers,

    Thomas


    Thomas Sondergaard Guest

  4. #4

    Default Re: When threads block

    On Tue, 23 Sep 2003 18:45:40 -0600, Hans Fugal wrote:
    > It's difficult to do any serious multi-threaded network programming when
    > the whole process blocks on system calls like read, write, and select. Is
    > this on a TODO list somewhere, or should I just accept it as a fact of
    > life? (and that may mean using a different language.)
    Ok, I jumped the gun just a little. On linux /most/ things don't block.
    But /some/ things do. e.g.

    thread = Thread.new do
    fifo = File.open("fifo","r")
    l = fifo.readline
    puts "#{Time.new} #{l}"
    end

    puts "#{Time.new} bar"
    thread.join

    Will block the entire process until the fifo is open. The following
    variation does not block the entire process, though:

    fifo = File.open("fifo","r")
    thread = Thread.new do
    l = fifo.readline
    puts "#{Time.new} #{l}"
    end

    puts "#{Time.new} bar"
    thread.join

    I say "most" because that's what the wise ones on #ruby-lang said, and
    indeed were surprised to see I had a counter example.

    Is there a list of these sorts of things? I need to know what to expect if
    I'm going to be able to use ruby for this (soft) real-time application.
    Hans Fugal Guest

  5. #5

    Default Re: When threads block

    il Wed, 24 Sep 2003 11:26:47 +0200, "Thomas Sondergaard"
    <thomasFirstNameGoesHereSondergaard.com> ha scritto::
    >> If you use Windows, it's a known problem but hard to fix with
    >> current thread implementation.
    >
    >When will we get a new thread implementation, then? :-)
    >
    >Seriously, could you outline how these things are implemented on other
    >platforms to avoid blocking the entire process? Why can't this trick be used
    >on the windows platform? I'd like to help if I can in any way.
    The problem should be related to windows' select().
    On *nix IO stuff is transparently put in a select, and ruby takes care
    of giving a little time to every thread when select gives them output.
    On windows this won't work cause select() just accepts socket
    descriptor. I wonder if these works in python.
    And is WaitForMultipleObjects() useful someway ?
    >Another question - is true thread safety on the TODO list for ruby? Oh and
    >this begs the third question: Is there a release plan for ruby? I would love
    >to know what kind of goodies I can expect for future releases?
    wait for Rite (ruby 2.0)


    gabriele renzi Guest

  6. #6

    Default Re: When threads block

    Hi,

    At Wed, 24 Sep 2003 22:38:58 +0900,
    Hans Fugal wrote:
    > Ok, I jumped the gun just a little. On linux /most/ things don't block.
    > But /some/ things do. e.g.
    >
    > thread = Thread.new do
    > fifo = File.open("fifo","r")
    > l = fifo.readline
    > puts "#{Time.new} #{l}"
    > end
    >
    > puts "#{Time.new} bar"
    > thread.join
    You can use IO::NONBLOCK with IO::RDONLY instead of "r".
    > Is there a list of these sorts of things? I need to know what to expect if
    > I'm going to be able to use ruby for this (soft) real-time application.
    File#flock, and some Socket methods do. Use resolv-replace.rb
    for the latter.

    --
    Nobu Nakada

    nobu.nokada@softhome.net Guest

  7. #7

    Default Re: When threads block

    Hello gabriele,

    Wednesday, September 24, 2003, 5:39:20 PM, you wrote:

    gr> The problem should be related to windows' select().
    gr> On *nix IO stuff is transparently put in a select, and ruby takes care
    gr> of giving a little time to every thread when select gives them output.
    gr> On windows this won't work cause select() just accepts socket
    gr> descriptor. I wonder if these works in python.
    gr> And is WaitForMultipleObjects() useful someway ?

    It's a joke right ?

    Every serious windows programming books warns you to not use 'select'
    on a window plattform. Using WASSelect is the normal way. It is much
    better then the UNIX API and integrates perfect into the windows
    environment. But a new implementation should not use them either.
    The best is to use asynchron system calls that calls back when data
    is available, it is much faster then the LINUX ip stack implementation.

    --
    Best regards,
    Lothar mailto:mailinglistsscriptolutions.com


    Lothar Scholz Guest

  8. #8

    Default Re: resolv-replace.rb bug? [WAS: When thread block]

    Hi,

    At Thu, 25 Sep 2003 09:24:45 +0900,
    Nathaniel Talbott wrote:
    > Is this intentional?
    >
    > irb(main):001:0> require 'socket'
    > => true
    > irb(main):002:0> TCPServer::new(55)
    > => #<TCPServer:0x2b9ef50>
    > irb(main):003:0> require 'resolv-replace'
    > => true
    > irb(main):004:0> TCPServer::new(55)
    > ArgumentError: wrong number of arguments(1 for 2)
    > from (irb):4:in `new'
    > from (irb):4
    No, resolv-replace.rb seems not to be 1.8 compliant.
    > Because of this, using DRb with resolv-replace doesn't work very well. Any
    > idea as to what is wrong?
    Does this work?


    Index: lib/resolv-replace.rb
    ================================================== =================
    RCS file: /cvs/ruby/src/ruby/lib/resolv-replace.rb,v
    retrieving revision 1.1
    diff -u -2 -p -r1.1 resolv-replace.rb
    --- lib/resolv-replace.rb 30 May 2001 09:10:26 -0000 1.1
    +++ lib/resolv-replace.rb 25 Sep 2003 01:49:23 -0000
    -1,2 +1,3
    +require 'socket'
    require 'resolv'

    -4,5 +5,5 class BasicSocket
    alias original_resolv_send send
    def send(mesg, flags, *rest)
    - rest[0] = Resolv.getaddress(rest[0]).to_s if 0 < rest.length
    + rest[0] = Resolv.getaddress(rest[0]).to_s unless rest.empty?
    original_resolv_send(mesg, flags, *rest)
    end
    -16,17 +17,18 class << IPSocket
    end

    -class << TCPSocket
    - alias original_resolv_new new
    - def new(host, service)
    - original_resolv_new(Resolv.getaddress(host).to_s, service)
    - end
    -
    - alias original_resolv_open open
    - def open(host, service)
    - original_resolv_open(Resolv.getaddress(host).to_s, service)
    +class TCPSocket
    + alias original_resolv_initialize initialize
    + def initialize(host, serv, *rest)
    + rest[0] = Resolv.getaddress(rest[0]).to_s unless rest.empty?
    + original_resolv_initialize(Resolv.getaddress(host) .to_s, serv, *rest)
    end
    end

    class UDPSocket
    + alias original_resolv_bind bind
    + def bind(host, port)
    + original_resolv_bind(Resolv.getaddress(host).to_s, port)
    + end
    +
    alias original_resolv_connect connect
    def connect(host, port)
    -36,6 +38,13 class UDPSocket
    alias original_resolv_send send
    def send(mesg, flags, *rest)
    - rest[0] = Resolv.getaddress(rest[0]).to_s if 0 < rest.length
    + rest[0] = Resolv.getaddress(rest[0]).to_s unless rest.empty?
    original_resolv_send(mesg, flags, *rest)
    end
    end
    +
    +class SOCKSSocket
    + alias original_resolv_initialize initialize
    + def initialize(host, serv)
    + original_resolv_initialize(Resolv.getaddress(host) .to_s, port)
    + end
    +end if defined? SOCKSSocket


    --
    Nobu Nakada

    nobu.nokada@softhome.net Guest

  9. #9

    Default Re: When threads block

    il Thu, 25 Sep 2003 09:15:50 +0900, Lothar Scholz
    <mailinglistsscriptolutions.com> ha scritto::
    >gr> And is WaitForMultipleObjects() useful someway ?
    >
    >It's a joke right ?
    >
    >Every serious windows programming books warns you to not use 'select'
    >on a window plattform.
    That's sure. I never ever read a windows programming book, this may
    matter :)
    Actually, I just wrote some win code for an OS course and at most used
    3 syscall ..
    >Using WASSelect is the normal way.
    Is'nt WSASelect even stuck to FD_SET==array of sockets?
    > It is much
    >better then the UNIX API and integrates perfect into the windows
    >environment.
    Sure it does, but well, ruby is mostly developed on unix :/


    anyway, I was simply referring to Message-Id:
    <200309240558.h8O5wE2s021288sharui.nakada.kanuma. tochigi.jp>
    gabriele renzi Guest

  10. #10

    Default Re: resolv-replace.rb bug? [WAS: When thread block]

    In article <200309250150.h8P1ow2s008367sharui.nakada.kanuma. tochigi.jp>,
    [email]nobu.nokadasofthome.net[/email] writes:
    > No, resolv-replace.rb seems not to be 1.8 compliant.
    Please commit it.
    --
    Tanaka Akira

    Tanaka Akira Guest

  11. #11

    Default Re: When threads block

    Hello gabriele,
    >>Using WASSelect is the normal way.
    gr> Is'nt WSASelect even stuck to FD_SET==array of sockets?

    No. Its working with GUI messages.
    >> It is much
    >>better then the UNIX API and integrates perfect into the windows
    >>environment.
    gr> Sure it does, but well, ruby is mostly developed on unix :/

    But it's a "very high level language" and it fallsback to extremely
    low level plattform dependent network code. That's a design decision i don't
    understand. It's okay to wrap this, but higher modules like the HTTP,
    FTP protocol shouldn't be based on a 'select' like model.


    --
    Best regards,
    Lothar mailto:mailinglistsscriptolutions.com


    Lothar Scholz Guest

Similar Threads

  1. Threads falling apart
    By Robert Klemme in forum Ruby
    Replies: 2
    Last Post: November 24th, 05:49 AM
  2. Sockets and threads
    By Mark J. Reed in forum Ruby
    Replies: 1
    Last Post: August 26th, 07:38 AM
  3. Using threads to obtain a value
    By Zachary P. Landau in forum Ruby
    Replies: 2
    Last Post: July 17th, 01:00 AM
  4. Max worker threads
    By Vinodk in forum Microsoft SQL / MS SQL Server
    Replies: 8
    Last Post: July 3rd, 06:44 PM
  5. Replies: 0
    Last Post: January 8th, 08:52 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139