Professional Web Applications Themes

FreeBSD 5.3: Sharedlibs using sharedlibs (and Tcl) - FreeBSD

Hi all, my question is, which references to other sharedlibs need to be in a shared library? When installing postgresql database system and the Tcl interface tool "pgaccess", I noticed that the kerberos support did not work. I installed postgresql 7.4.5 from the ports colletion as of RELEASE 5.3. This is not the newest, so I did not create a bugreport, but instead figured out the problem (and a solution) by myself (with some support from the pgaccess user community). But now I would like to *understand* what was going wrong and why I could fix it the way I ...

  1. #1

    Default FreeBSD 5.3: Sharedlibs using sharedlibs (and Tcl)

    Hi all,

    my question is,
    which references to other sharedlibs need to be in a shared
    library?

    When installing postgresql database system and the Tcl interface tool
    "pgaccess", I noticed that the kerberos support did not work.

    I installed postgresql 7.4.5 from the ports colletion as of RELEASE
    5.3. This is not the newest, so I did not create a bugreport, but
    instead figured out the problem (and a solution) by myself (with
    some support from the pgaccess user community).

    But now I would like to *understand* what was going wrong and why
    I could fix it the way I did.

    I describe the fabric:
    1. postgresql brings a library "libpq.so.3" into /usr/local/lib. This
    library contains all the code to access the database server.
    If we use kerberos, then this library will call functions from
    the bunch of kerberos libraries (libkrb5, libasn1, etc etc)
    in /usr/lib.
    2. postgresql also brings another, optional library "libpgtcl.so.2"
    into /usr/local/lib. This library contains special function for
    accessing the database server from Tcl.
    This library calls the functions in libpq.so.3.
    3. pgaccess is a Tcl script. It wants to load libpgtcl.so. It finds
    and loads libpgtcl.so, it finds and loads the necessary functions
    from libpq.so, and then, if we have kerberos compiled in, it
    recognizes one of the needed kerberos functions, and complains
    that it cannot find this function referenced from libpq.so. So the
    load of libpgtcl.so fails.

    As far as I see, this problem does not arise with binaries. All
    binary progams using libpq.so do support kerberos, and it works.

    Then I noticed that sharedlibs contain a section where other needed
    sharedlibs can be explicitely mentioned. And I noticed that
    libpgtcl.so contains such a mentioning of libpq.so - so this is
    found by Tcl.
    But libqp.so does not contain an explicit mentioning of the kerberos
    libraries.
    So I tried around with the linker command until I practiced such
    an explicit mentioning into libpq.so.
    And then step 3 from above did succeed!

    I conclude:
    Since this is now a matter of how sharedlibs are built on the system,
    this does not only concern kerberos and postgresql, but concerns any
    component which shall be called from Tcl.

    I have now two versions of my libpq.so - both contain the same code,
    but one will support kerberos from Tcl, and the other (the one that
    was built in the standard way) will not.
    The only difference between both shows up in the output of "readelf
    -a" as follows:

    The standard build that does not work:
    -------------------------------------------------------
    [...]
    Dynamic segment at offset 0x19774 contains 21 entries:
    Tag Type Name/Value
    0x00000001 (NEEDED) Shared library: [libintl.so.6]
    0x00000001 (NEEDED) Shared library: [libssl.so.3]
    0x00000001 (NEEDED) Shared library: [libcrypto.so.3]
    0x0000000e (SONAME) Library soname: [libpq.so.3]
    0x0000000f (RPATH) Library rpath: [/usr/local/lib]
    [...]

    My modified build that does work:
    -------------------------------------------------------
    [...]
    Dynamic segment at offset 0x19774 contains 26 entries:
    Tag Type Name/Value
    0x00000001 (NEEDED) Shared library: [libintl.so.6]
    0x00000001 (NEEDED) Shared library: [libssl.so.3]
    0x00000001 (NEEDED) Shared library: [libcrypto.so.3]
    0x00000001 (NEEDED) Shared library: [libkrb5.so.7]
    0x00000001 (NEEDED) Shared library: [libasn1.so.7]
    0x00000001 (NEEDED) Shared library: [libroken.so.7]
    0x00000001 (NEEDED) Shared library: [libcrypt.so.2]
    0x00000001 (NEEDED) Shared library: [libcom_err.so.2]
    0x0000000e (SONAME) Library soname: [libpq.so.3]
    0x0000000f (RPATH) Library rpath: [/usr/local/lib]
    [...]

    So, my question now is: where is the conceptional error which led
    to the software not working at first?
    In Tcl? In the linker? In the system loader? In the build environment
    (port)? In the postgresql makefiles? In the FreeBSD sharedlib
    management? In kerberos? Or somewhere else?

    And this seems complex enough to me so I do not even know how
    to search if it might be a known bug that has already been fixed
    in the meantime...

    PMc
    Peter Guest

  2. #2

    Default Re: FreeBSD 5.3: Sharedlibs using sharedlibs (and Tcl)

    Hi Peter,

    There is (and was in 5.3 release) a knob to build postgresql with Kerberos.
    WITH_HEIMDAL_KRB5=YES. Did you try that when building PostgreSQL? It would
    probably do the same thing as you managed by trying around with the linker
    command. Setting this knob to yes will add --with-krb5=/usr to the
    configure arguments, and this will trigger stuff in postgresql makefiles
    and possibly also additional code.

    I think this is what went wrong; the postgresql source has a configure
    option for building and linking with Kerberos, this option is reflected in
    the port, but it was not used in this case.

    Best regards,
    Palle



    --On tisdag, mars 15, 2005 18.06.56 +0100 Peter Much
    <dinoex.sub.org> wrote:
     




    Palle Guest

  3. #3

    Default Re: FreeBSD 5.3: Sharedlibs using sharedlibs (and Tcl)

    On Wed, Mar 16, 2005 at 03:35:16PM +0100, Palle Girgensohn wrote:
    !
    ! --On onsdag, mars 16, 2005 11.43.31 +0100 Peter Much
    ! <dinoex.sub.org> wrote:
    !
    ! >! So, you're saying that pctclsh *can* access, but pgaccess *cannot*?
    ! >Odd... ! I would expect they'd both use the same lib to connect, no?
    ! >I'll have to
    ! >
    ! >They use the same libraries, yes. But Tcl interpreter seem to need more
    ! >advice on where to find sub-functions in other libraries. It looks
    ! >like this:
    ! >
    ! >pgtclsh -finds-> libpgtcl.so -finds-> libpq.so -finds-> libkrb5.so
    ! >pgaccess -loads-> libpgtcl.so -finds-> libpq.so -fails-> libkrb5.so
    !
    ! Uh, OK. I'm not qualified enough with linkers to answer this, I'm afraid.
    ! Did you try the pgsql-interfaces mailing list?

    Oh well, same with me. I sent a copy of one of my reports to that
    list, yes. But only got feedback that it will be evaluated by
    moderator, as I am not signed on that list.
    I'm actually no professional psql user - the database is just a small
    part of my installation, mainly logging the lowlevel error counts
    from my exabyte drives and providing reports about tape wearout.
    And the kerberos is just there for fun, as a reference installation.

    Nevertheless, I would think this is not a matter for the postgres
    community. Because this would happen the same way with any other
    application that provides Tcl support and kerberos support (or maybe
    also with other components of the system, if they are used from Tcl).

    So it seems either a Tcl problem or a linker/loader problem. Which,
    I cannot say - maybe both.

    ! >And then I found that it is enough to place into libpq.so the explicit
    ! >references to libkrb5 and the other kerberos libs. That is what the
    ! >"readelf -a" output in my other mail shows.
    !
    ! sounds like a better solution, yes... Shouldn't they always be there?
    ! Sounds like a bug to me?

    Thats the question.
    I just did a little more investigation (like reading manpages)
    and found out _WHY_ it does work for pgtclsh but not for pgaccess.

    There is a command "ldd" that shows nested library dependencies
    for any program.
    For pgtclsh it shows all the kerberos libs:

    bash-3.00# ldd /usr/local/bin/pgtclsh
    /usr/local/bin/pgtclsh:
    libpgtcl.so.2 => /usr/local/lib/libpgtcl.so.2 (0x28075000)
    libpq.so.3 => /usr/local/lib/libpq.so.3 (0x2807d000)
    libtcl84.so.1 => /usr/local/lib/libtcl84.so.1 (0x28097000)
    libm.so.3 => /lib/libm.so.3 (0x28135000)
    libkrb5.so.7 => /usr/lib/libkrb5.so.7 (0x2814f000)
    libasn1.so.7 => /usr/lib/libasn1.so.7 (0x28186000)
    libcrypto.so.3 => /lib/libcrypto.so.3 (0x281a6000)
    libroken.so.7 => /usr/lib/libroken.so.7 (0x2829b000)
    libcrypt.so.2 => /lib/libcrypt.so.2 (0x282a9000)
    libcom_err.so.2 => /usr/lib/libcom_err.so.2 (0x282c1000)
    libz.so.2 => /lib/libz.so.2 (0x282c3000)
    libreadline.so.5 => /lib/libreadline.so.5 (0x282d3000)
    libutil.so.4 => /lib/libutil.so.4 (0x282ff000)
    libc.so.5 => /lib/libc.so.5 (0x2830b000)
    libintl.so.6 => /usr/local/lib/libintl.so.6 (0x283e4000)
    libssl.so.3 => /usr/lib/libssl.so.3 (0x283ed000)
    libncurses.so.5 => /lib/libncurses.so.5 (0x2841b000)
    libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x2845a000)

    But for libpgtcl.so (this is the first elf binary that pgaccess gets
    to see) it does not show these kerberos libraries (I use the old
    libpq.so here, not the one that I have modified):

    bash-3.00# ldd /usr/local/lib/libpgtcl.so
    /usr/local/lib/libpgtcl.so:
    libpq.so.3 => /usr/local/lib/libpq.so.3 (0x2815a000)
    libtcl84.so.1 => /usr/local/lib/libtcl84.so.1 (0x28174000)
    libm.so.3 => /lib/libm.so.3 (0x28212000)
    libintl.so.6 => /usr/local/lib/libintl.so.6 (0x2822c000)
    libssl.so.3 => /usr/lib/libssl.so.3 (0x28235000)
    libcrypto.so.3 => /lib/libcrypto.so.3 (0x28263000)
    libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x28358000)

    Then the explanation became simple: these kerberos libraries get
    just LITERALLY LISTED WITHIN THE pgtclsh BINARY! And this is an
    impossible method for a Tcl script.

    bash-3.00# readelf -d /usr/local/bin/pgtclsh | grep krb5
    0x00000001 (NEEDED) Shared library: [libkrb5.so.7]

    So now we have a full explanation for the behaviour, but not really
    a solution.
    Instead, this looks like a fundamental question about how to load
    nested elf sharedlibs from interpreter languages.
     
    would be: every shared library must reference all other shared
    libraries from which it uses functions. The shared library cannot
    rely on the executable to do this job, because the executable
    may be an interpreter script, which neither is able to do this
    nor would it want to know them all.
     
    is defective. So You were right and its a problem for the
    postgresql developers.

    But as I am not competent with shared libraries and development
    systems and such stuff, I would very much appreciate the opinion
    of somebody with profound knowledge of these things. And anyway,
    perl should have similar problems.

    Ok, lets see - which nice mailinglists do we crosspost today?
    I go with questions and ports, and add hackers. I think we're now
    deep enough in the rabbithole for them. ;-)
    And psql-interfaces.

    PMc
    Peter Guest

  4. #4

    Default Re: FreeBSD 5.3: Sharedlibs using sharedlibs (and Tcl)

    --On onsdag, mars 16, 2005 20.13.03 +0100 Peter Much
    <dinoex.sub.org> wrote:
     


    Hi!

    Has anyone been able to shed any light on this? postgres + kerberos makes
    libpqxx and libpqtcl fail, at least on FreeBSD. Any ideas?

    /Palle

    Palle Guest

Similar Threads

  1. FreeBSD
    By switchblade3i3 in forum Macromedia Flash Flashcom
    Replies: 2
    Last Post: March 16th, 02:21 AM
  2. Compile FreeBSD RELENG_5 on FreeBSD 4-STABLE
    By Brovo Karokin in forum FreeBSD
    Replies: 1
    Last Post: February 25th, 09:04 AM
  3. Instead of freebsd.com, why not...
    By Joshua Tinnin in forum FreeBSD
    Replies: 3
    Last Post: February 21st, 06:20 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139