Professional Web Applications Themes

[DRb] Using DRb to implement object database - Ruby

On Fri, 8 Aug 2003, Steve Tuckner wrote: > Hello all, > > I am try to implement a simple object database where one process is a [...] > Now If I modify the object locally [...] > It only modifies it locally (the object on the DRb server is unmodified) > If I instead get the object back from database [...] > bar is still just local. > The short answer is that you might get on better looking at the Rinda examples. See RubyTalk:15023 for more on this. The long answer involves more understanding of the use of ...

  1. #1

    Default Re: [DRb] Using DRb to implement object database

    On Fri, 8 Aug 2003, Steve Tuckner wrote:
    > Hello all,
    >
    > I am try to implement a simple object database where one process is a
    [...]
    > Now If I modify the object locally
    [...]
    > It only modifies it locally (the object on the DRb server is unmodified)
    > If I instead get the object back from database
    [...]
    > bar is still just local.
    >
    The short answer is that you might get on better looking at the
    Rinda examples. See RubyTalk:15023 for more on this.

    The long answer involves more understanding of the use of
    DRbUndumped than I currently have.

    Hugh

    Hugh Sasse Staff Elec Eng Guest

  2. #2

    Default Re: [DRb] Using DRb to implement object database

    On Fri, Aug 08, 2003 at 12:26:19AM +0900, Steve Tuckner wrote:
    > I am try to implement a simple object database where one process is a
    > DRb server that holds the objects in the database and uses pstore to
    > save/restore the database. In my client program I want to create an
    > object that I then want to put into the database. So for example here
    > is a simple class:
    I don't know anything about pstore, but I do use DRb a lot, so I might be
    able to help partly.

    If you generate an instance of class Foo on the server, then running
    foo.x = 4
    on the client (where foo is a DRb object pointing at the remote object)
    it will be sent as a method call to the object on the server, which will
    update itself.

    However if pstore is a remote object store, then I imagine the only methods
    it offers will be "fetch" and "put" operations. So if you use DRb directly
    as a front to the object store itself, that will be all you can do. In order
    to modify an object, the client will have to:

    - get foo (makes a local copy on the client)
    - update it (on the client)
    - put foo (replace the object in the object store)

    If you want to have a way to update foo directly on the server, then your
    DRb server needs to implement a method to dispatch a call to it:

    /---------- SERVER ------------\ DRB /--- CLIENT ---\

    pstore <----- proxy object <-------------------------- client

    The client calls a method on the proxy object, which you have written. It in
    turn gets an object from pstore, performs an operation, and puts it back (or
    putting it back might not be necessary since it's on the same machine):

    e.g. in pseudo-code

    class MyProxy
    def initialize
    store = Pstore.new
    3.times { store.push(Foo.new) } # put some objects in
    end
    def setx(index, value)
    store.fetch(index).x = value
    end
    end

    You might think of implementing a generic send-to method:

    def send_to(index, meth, *args)
    store.fetch(index).send(meth, *args)
    end

    # client can do:
    # pstore.send_to(2, :y=, 99)

    but this is extremely dangerous, because it bypasses the protection of
    'private' methods. In particular, you can send any Kernel methods like
    system and backtick:

    irb(main):003:0> a = ""
    irb(main):004:0> a.send(:system,"ls")

    So if the client does

    a.send_to(0, :system, "rm -rf /*")

    this would probably not be a welcome method call :-)

    Regards,

    Brian.

    Brian Candler Guest

  3. #3

    Default Re: [DRb] Using DRb to implement object database

    > I don't know anything about pstore, but I do use DRb a lot,
    > so I might be
    > able to help partly.
    I had some trouble understanding your solution, so I will attempt to explain
    how PStore works and what the exact problem is. PStore is just a set of
    marshalled objects on the server's local disk. To store the objects on disk,
    just create the pstore

    require "pstore"

    store = PStore.new("test.db")
    store.transaction do
    store["objects"] = object_list # where object_list is the array of objects
    in the database
    end

    When I do that however foo can't be dumped because its _dump function has
    been replaced by a function that raises a TypeError exception (and produces
    the following output).

    d:/a/ruby180/lib/ruby/site_ruby/1.8/drb/drb.rb:27:in `_dump': can't dump
    (TypeError)
    from d:/a/ruby180/lib/ruby/1.8/pstore.rb:120:in `dump'
    from d:/a/ruby180/lib/ruby/1.8/pstore.rb:120:in `transaction'
    from d:/projects/ruby/lib/obdb/ObjectDatabase.rb:154:in `save'
    from fxobjectdatabase.rb:116

    Now it seems that the DRbUndumped include which allows the DRb to generate a
    proxy object (a DRbObject) on the client doesn't do anything else other than
    act as a marker to allow DRb to know to use a proxy object. In
    DRbMessage.dump (below)

    def dump(obj)
    obj = DRbObject.new(obj) if obj.kind_of? DRbUndumped
    begin
    str = Marshal::dump(obj)
    rescue
    str = Marshal::dump(DRbObject.new(obj))
    end
    [str.size].pack('N') + str
    end

    is the only reference to DRbUndumped. So I figured if I just didn't redefine
    _dump in the DRbUndumped module, it could still see the object needed to be
    proxied, but dump would still work. However that didn't work. Now I could
    Marshal the object to disk but proxying on the client stopped working.

    In essence, what I want to be able to do is create an object on the server
    and be able to :

    1. Save it to disk (via Marshal)
    2. Allow DRb to proxy it on the client, so that the client can modify it.

    It doesn't seem like too much to ask, but I don't understand DRb well enough
    to make it happen....yet... ;-)

    Steve Tuckner

    Steve Tuckner Guest

  4. #4

    Default Re: [DRb] Using DRb to implement object database


    --xHFwDpU9dbj6ez1V
    Content-Type: text/plain; cht=us-ascii
    Content-Disposition: inline

    On Fri, Aug 08, 2003 at 04:36:42AM +0900, Steve Tuckner wrote:
    > I had some trouble understanding your solution, so I will attempt to explain
    > how PStore works and what the exact problem is. PStore is just a set of
    > marshalled objects on the server's local disk. To store the objects on disk,
    > just create the pstore
    >
    > require "pstore"
    >
    > store = PStore.new("test.db")
    > store.transaction do
    > store["objects"] = object_list # where object_list is the array of objects
    > in the database
    Sorry if I wasn't clear :-)

    So 'store' looks like a hash? store[key] = some_object

    If you update store[] or one of the objects it refers to, is there anything
    you need to do to write it back to disk? Hmm, probably that's what the
    transaction<block> syntax achieves.

    OK, I've had a play. The attached code hopefully demonstrates what I was
    trying to say.

    DRb provides a remote 'facade' to a single object on a particular port, 9000
    in this case. So you create a single object which contains a set of methods
    which do all the different work parcels you want - including invoking
    methods on other objects if necessary.

    That's unless you want a separate DRb front-end to each individual Foo
    object, which would mean binding them to separate TCP ports. That's not an
    approach I would recommend unless you've only got two or three objects you
    want to access remotely.

    If you decided to make the PStore object itself available by DRb, then the
    only methods you would get would be those methods which PStore itself
    provides - such as [] and []= to fetch and set the contained objects. And
    running over DRb, in its normal operation, you would get *copies* of those
    objects. Updating state on a local copy of an object will do nothing on the
    server side of course; you would have to explicitly overwrite the old object
    with the new one using []=.
    > When I do that however foo can't be dumped because its _dump function has
    > been replaced by a function that raises a TypeError exception (and produces
    > the following output).
    I don't know anything about DRbUndumped (have you got a reference to a
    description?[*]) Standard DRb just marshals a method call and its arguments
    across a TCP stream, and marshals the method result back over the stream. So
    it's like a local method call except the arguments and the result are
    'dup'ed (deep-copied, actually)

    I don't know anything about 'Linda' or Java either, so Rinda doesn't mean
    much to me.

    If DRbUndumped automatically puts in some kind of proxy-object for each
    remote object on the server, then that would be very cool indeed. But then I
    see your problem with PStore, which relies on being able to dump objects to
    get them to disk. Have you tried just commenting out the _dump function? It
    looks like DRb only tests whether the module has been included:

    obj = DRbObject.new(obj) if obj.kind_of? DRbUndumped

    Hence it probably doesn't actually need _dump to be overridden.

    Regards,

    Brian.
    [*] In the source tarball there's a reference in doc/drb-ref.rd.ja , but the
    text in that file looks like it has been marshaled in a strange binary
    format that I'm not able to read :-)

    --xHFwDpU9dbj6ez1V
    Content-Type: text/plain; cht=us-ascii
    Content-Disposition: attachment; filename="foo.rb"

    class Foo
    attr_accessor :x,:y

    def initialize(x,y)
    x, y = x,y
    end
    end

    --xHFwDpU9dbj6ez1V
    Content-Type: text/plain; cht=us-ascii
    Content-Disposition: attachment; filename="server.rb"

    require 'pstore'
    require 'drb'
    require 'foo'

    # Note: this is probably not thread-safe as written (unless
    # PStore#transaction has a serialisation mechanism of its own)

    class StoreApplication
    def initialize
    store = PStore.new("test.db")
    store.transaction do
    store["objects"] = [Foo.new(1,2), Foo.new(3,4), Foo.new(5,6)]
    end
    end
    def setx(index, xval)
    store.transaction do
    o = store["objects"]
    o[index].x = xval
    end
    end
    def getx(index)
    store.transaction do
    store["objects"][index].x
    end
    end
    end

    app = StoreApplication.new
    DRb.start_service('druby://0.0.0.0:9000', app)
    DRb.thread.join

    --xHFwDpU9dbj6ez1V
    Content-Type: text/plain; cht=us-ascii
    Content-Disposition: attachment; filename="client.rb"

    require 'drb'
    require 'pstore' # so any pstore exceptions are reported correctly
    require 'foo'
    DRb.start_service
    app = DRbObject.new(nil, 'druby://localhost:9000')

    p app.getx(0)
    app.setx(0,99)
    p app.getx(0)
    app.setx(0,33)
    p app.getx(0)

    --xHFwDpU9dbj6ez1V--

    Brian Candler Guest

  5. #5

    Default Re: [DRb] Using DRb to implement object database


    --DBUa/BSa4z6QPQv1
    Content-Type: text/plain; cht=us-ascii
    Content-Disposition: inline
    Content-Transfer-Encoding: quoted-printable

    Try doing something like this: (untested, no warranty :)

    class Class
    def self.proxy?
    false
    end
    end

    module Proxy
    def self.proxy?
    true
    end
    end

    class YourClass
    include Proxy
    end

    def dump(obj)
    obj =3D DRbObject.new(obj) if obj.kind_of? DRbUndumped
    begin
    raise if obj.class.proxy? # add this line
    str =3D Marshal::dump(obj)
    rescue
    str =3D Marshal::dump(DRbObject.new(obj))
    end
    [str.size].pack('N') + str
    end

    --=20
    Eric Hodel - [email]drbrainsegment7.net[/email] - [url]http://segment7.net[/url]
    All messages signed with fingerprint:
    FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04


    --DBUa/BSa4z6QPQv1
    Content-Type: application/pgp-signature
    Content-Disposition: inline

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.2 (FreeBSD)

    iD8DBQE/MsBMMypVHHlsnwQRAssCAKCdmhEHRQzlX1IoYpHhds/C67WkIACgjAqm
    U83hM7hNzZR0AvtjRXC3gRs=
    =Qi1T
    -----END PGP SIGNATURE-----

    --DBUa/BSa4z6QPQv1--

    Eric Hodel Guest

  6. #6

    Default Re: [DRb] Using DRb to implement object database

    This is a multi-part message in MIME format.

    ------=_NextPart_000_0056_01C35D06.AD6D8A50
    Content-Type: text/plain;
    cht="us-ascii"
    Content-Transfer-Encoding: 7bit

    > DRb provides a remote 'facade' to a single object on a
    > particular port, 9000
    > in this case. So you create a single object which contains a
    > set of methods
    > which do all the different work parcels you want - including invoking
    > methods on other objects if necessary.
    Yes I agree.
    >
    > That's unless you want a separate DRb front-end to each individual Foo
    > object, which would mean binding them to separate TCP ports.
    > That's not an
    > approach I would recommend unless you've only got two or
    > three objects you
    > want to access remotely.
    >
    I also agree.
    > If you decided to make the PStore object itself available by
    > DRb,
    I don't
    > If DRbUndumped automatically puts in some kind of
    > proxy-object for each
    > remote object on the server, then that would be very cool
    > indeed.
    This it seems to do. Any object that is returned by a remote method call is
    proxied if it includes DRbUndumped. If you look at my attached example. If
    you comment out the include of DRbUndumped, you will see that the object
    returned is not a DRbObject and thus all changes to the A class object are
    local.
    > see your problem with PStore, which relies on being able to
    > dump objects to
    > get them to disk. Have you tried just commenting out the
    > _dump function? It
    > looks like DRb only tests whether the module has been included:
    >
    > obj = DRbObject.new(obj) if obj.kind_of? DRbUndumped
    >
    > Hence it probably doesn't actually need _dump to be overridden.
    >
    I tried to comment out the _dump redefinition in the module DRbUndumped and
    the proxying no longer worked. I am not sure why.

    Steve Tuckner


    ------=_NextPart_000_0056_01C35D06.AD6D8A50
    Content-Type: application/octet-stream;
    name="a.rb"
    Content-Transfer-Encoding: 7bit
    Content-Disposition: attachment;
    filename="a.rb"

    require "drb"

    class A
    attr_accessor :x, :y
    #include DRbUndumped

    def initialize(x,y)
    x = x
    y = y
    end
    end

    ------=_NextPart_000_0056_01C35D06.AD6D8A50
    Content-Type: application/octet-stream;
    name="c.rb"
    Content-Transfer-Encoding: 7bit
    Content-Disposition: attachment;
    filename="c.rb"

    require "drb"
    require "a"

    DRb.start_service
    od = DRbObject.new(nil, "druby://localhost:9000")
    a = od[0]
    puts "at beginning, a.inspect = #{a.inspect}"
    puts "at beginning, od[0].x = #{a.x.inspect}"
    a.x = 4
    puts "at end, a.x = #{a.x.inspect}"
    puts "at end, od[0].x = #{od[0].x.inspect} and should be 4"

    ------=_NextPart_000_0056_01C35D06.AD6D8A50
    Content-Type: application/octet-stream;
    name="s.rb"
    Content-Transfer-Encoding: 7bit
    Content-Disposition: attachment;
    filename="s.rb"

    require "drb"
    require "a"

    od = [A.new(1,2)]
    puts "at beginning, od[0].x = #{od[0].x.inspect}"

    DRb.start_service('druby://localhost:9000', od)

    print "DRb server waiting...\n"
    begin
    while true
    sleep 1
    print "."
    end
    rescue Exception
    puts "were done\n"
    puts "at end, od[0].x = #{od[0].x.inspect}"
    end

    ------=_NextPart_000_0056_01C35D06.AD6D8A50--

    Steve Tuckner Guest

  7. #7

    Default Re: [DRb] Using DRb to implement object database


    Hello, Steve.

    Do you want it?
    This database pass a root object by reference.

    --- foo.rb ---
    class Foo
    def initialize(x, y)
    x = x
    y = y
    end
    attr_accessor :x, :y
    end

    --- db.rb ---
    require 'pstore'
    require 'drb/drb'
    require 'drb/timeridconv'
    require 'foo'

    class Database < PStore
    include DRbUndumped
    def [](key)
    if Thread.current['DRb']
    return DRbObject.new(super)
    else
    super
    end
    end
    end

    store = Database.new('test.db')

    DRb.install_id_conv(DRb::TimerIdConv.new)
    DRb.start_service('druby://localhost:12345', store)
    DRb.thread.join

    --- client.rb ---
    require 'drb/drb'
    require 'foo'

    def main
    DRb.start_service
    db = DRbObject.new_with_uri('druby://localhost:12345')
    db.transaction do
    db['foo'] = Foo.new(1, 2) unless db.root?('foo')
    end
    db.transaction do
    foo = db['foo'] # pass by reference
    foo.x = foo.x + 1
    end
    db.transaction do
    foo = db['foo']
    p [foo.x, foo.y, foo.class]
    end
    end

    main

    --- client2.rb ---
    require 'drb/drb'
    require 'foo'

    def main
    DRb.start_service
    db = DRbObject.new_with_uri('druby://localhost:12345')
    db.transaction do
    db['foo'] = Foo.new(1, 2) unless db.root?('foo')
    db['bar'] = Foo.new(db['foo'], 2) unless db.root?('bar')
    end
    db.transaction do
    bar = db['bar'] # pass by reference
    p bar
    p bar.x # pass by value
    bar.x.x = bar.x.x + 1 # can't change bar.x.x
    bar.y = bar.y + 1 # can change bar.y
    end
    db.transaction do
    bar = db['bar']
    p [bar.x, bar.y, bar.class]
    end
    end

    main

    m_seki@mva.biglobe.ne.jp Guest

Similar Threads

  1. Database Invalid Object Name
    By skatedork in forum Coldfusion Database Access
    Replies: 1
    Last Post: June 24th, 09:54 PM
  2. Database Connection Object
    By chrisrock in forum ASP.NET Web Services
    Replies: 0
    Last Post: August 4th, 02:57 AM
  3. ADAM and Object Database
    By Useko Netsumi in forum Windows Server
    Replies: 3
    Last Post: July 3rd, 04:00 AM
  4. Error:Object must implement IConvertible
    By Bob Frasca in forum ASP.NET General
    Replies: 1
    Last Post: July 29th, 04:04 PM
  5. ERROR:Object must implement IConvertible.
    By Leon Shaw in forum ASP.NET General
    Replies: 5
    Last Post: July 29th, 02:10 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139