process using .so files created with different file format

[qazmlp]

A process uses some shared libraries(.so files). We are suspecting
stack corruption problem as a value returned from one particular
function to its caller is found to be different.The return value is
printed once inside the function body, just before the 'return'
statement and immediately again after the function call.


Though there can be many reasons for this problem, we are suspecting
on
the file format of the files with which the .so file were created.
When we opened the file, vim shows that, some of the files that are
required tp
create a .so are in 'DOS' format. But, we are compiling in an
Unix(Solaris) machine using SUN C++ compiler. The process also runs on
Solaris only.Can that cause any problems(may not be 'stack
corruption', but something else)?

[qazmlp]

A process uses some shared libraries(.so files). We are suspecting
stack corruption problem as a value returned from one particular
function to its caller is found to be different.The return value is
printed once inside the function body, just before the 'return'
statement and immediately again after the function call.


Though there can be many reasons for this problem, we are suspecting
on
the file format of the files with which the .so file were created.
When we opened the file, vim shows that, some of the files that are
required tp
create a .so are in 'DOS' format. But, we are compiling in an
Unix(Solaris) machine using SUN C++ compiler. The process also runs on
Solaris only.Can that cause any problems(may not be 'stack
corruption', but something else)?

[Sven Gohlke]

qazmlp wrote:

> A process uses some shared libraries(.so files). We are suspecting
> stack corruption problem as a value returned from one particular
> function to its caller is found to be different.The return value is
> printed once inside the function body, just before the 'return'
> statement and immediately again after the function call.

That's the definition of stack corruption, isn't it?

> Though there can be many reasons for this problem, we are suspecting
> on
> the file format of the files with which the .so file were created.
> When we opened the file, vim shows that, some of the files that are
> required tp
> create a .so are in 'DOS' format.

COFF isn't a DOS format but a standardized linker format. Are the libraries
binary only third party products (and if so, what should they do)?

> But, we are compiling in an
> Unix(Solaris) machine using SUN C++ compiler. The process also runs on
> Solaris only.Can that cause any problems(may not be 'stack
> corruption', but something else)?

If the loader can load the library this shouldn't be the problem. (According
to the documentation to the GNU binutils the ld.so of SUN had a bug, but
let's hope it isn't _as_ buggy:-)
You amy have used different or dangerous compiler options (like
optimization, never use a '-O') while compiling Your modules.
--
Best Regards
Sven

[Sven Gohlke]

qazmlp wrote:

> A process uses some shared libraries(.so files). We are suspecting
> stack corruption problem as a value returned from one particular
> function to its caller is found to be different.The return value is
> printed once inside the function body, just before the 'return'
> statement and immediately again after the function call.

That's the definition of stack corruption, isn't it?

> Though there can be many reasons for this problem, we are suspecting
> on
> the file format of the files with which the .so file were created.
> When we opened the file, vim shows that, some of the files that are
> required tp
> create a .so are in 'DOS' format.

COFF isn't a DOS format but a standardized linker format. Are the libraries
binary only third party products (and if so, what should they do)?

> But, we are compiling in an
> Unix(Solaris) machine using SUN C++ compiler. The process also runs on
> Solaris only.Can that cause any problems(may not be 'stack
> corruption', but something else)?

If the loader can load the library this shouldn't be the problem. (According
to the documentation to the GNU binutils the ld.so of SUN had a bug, but
let's hope it isn't _as_ buggy:-)
You amy have used different or dangerous compiler options (like
optimization, never use a '-O') while compiling Your modules.
--
Best Regards
Sven

[qazmlp]

Sven Gohlke <sven@clio.in-berlin.de> wrote in message news:<2397021.Tnpv1LEklG@clio.in-berlin.de>...

> qazmlp wrote:
>

> > A process uses some shared libraries(.so files). We are suspecting
> > stack corruption problem as a value returned from one particular
> > function to its caller is found to be different.The return value is
> > printed once inside the function body, just before the 'return'
> > statement and immediately again after the function call.

>
> That's the definition of stack corruption, isn't it?
>

> > Though there can be many reasons for this problem, we are suspecting
> > on
> > the file format of the files with which the .so file were created.
> > When we opened the file, vim shows that, some of the files that are
> > required tp
> > create a .so are in 'DOS' format.

>
> COFF isn't a DOS format but a standardized linker format. Are the libraries
> binary only third party products (and if so, what should they do)?
>

> > But, we are compiling in an
> > Unix(Solaris) machine using SUN C++ compiler. The process also runs on
> > Solaris only.Can that cause any problems(may not be 'stack
> > corruption', but something else)?

>
> If the loader can load the library this shouldn't be the problem. (According
> to the documentation to the GNU binutils the ld.so of SUN had a bug, but
> let's hope it isn't _as_ buggy:-)
> You amy have used different or dangerous compiler options (like
> optimization, never use a '-O') while compiling Your modules.

That is the key here...I used '-O' flag. I have been using it for a
quite long time. Can you point out any bug in using that flag and the
possibilities of that being the cause of the problem that I am facing.

Thanks!

[qazmlp]

Sven Gohlke <sven@clio.in-berlin.de> wrote in message news:<2397021.Tnpv1LEklG@clio.in-berlin.de>...

> qazmlp wrote:
>

> > A process uses some shared libraries(.so files). We are suspecting
> > stack corruption problem as a value returned from one particular
> > function to its caller is found to be different.The return value is
> > printed once inside the function body, just before the 'return'
> > statement and immediately again after the function call.

>
> That's the definition of stack corruption, isn't it?
>

> > Though there can be many reasons for this problem, we are suspecting
> > on
> > the file format of the files with which the .so file were created.
> > When we opened the file, vim shows that, some of the files that are
> > required tp
> > create a .so are in 'DOS' format.

>
> COFF isn't a DOS format but a standardized linker format. Are the libraries
> binary only third party products (and if so, what should they do)?
>

> > But, we are compiling in an
> > Unix(Solaris) machine using SUN C++ compiler. The process also runs on
> > Solaris only.Can that cause any problems(may not be 'stack
> > corruption', but something else)?

>
> If the loader can load the library this shouldn't be the problem. (According
> to the documentation to the GNU binutils the ld.so of SUN had a bug, but
> let's hope it isn't _as_ buggy:-)
> You amy have used different or dangerous compiler options (like
> optimization, never use a '-O') while compiling Your modules.

That is the key here...I used '-O' flag. I have been using it for a
quite long time. Can you point out any bug in using that flag and the
possibilities of that being the cause of the problem that I am facing.

Thanks!

[Sven Gohlke]

qazmlp wrote:

>> You amy have used different or dangerous compiler options (like
>> optimization, never use a '-O') while compiling Your modules.

>
> That is the key here...I used '-O' flag. I have been using it for a
> quite long time. Can you point out any bug in using that flag and the
> possibilities of that being the cause of the problem that I am facing.

A library function may expect the return value in a register while the
caller returns it on the stack. Usual You get this problem only if You
compile parts with the optimizer and parts without.

There is a reason why You often read: "Never use the -O flag" (except,
maybe, the compiler is the GCC where even '-O2' works astonishingly
stable:-)
--
Best Regards
Sven

[Sven Gohlke]

qazmlp wrote:

>> You amy have used different or dangerous compiler options (like
>> optimization, never use a '-O') while compiling Your modules.

>
> That is the key here...I used '-O' flag. I have been using it for a
> quite long time. Can you point out any bug in using that flag and the
> possibilities of that being the cause of the problem that I am facing.

A library function may expect the return value in a register while the
caller returns it on the stack. Usual You get this problem only if You
compile parts with the optimizer and parts without.

There is a reason why You often read: "Never use the -O flag" (except,
maybe, the compiler is the GCC where even '-O2' works astonishingly
stable:-)
--
Best Regards
Sven

[Casper H.S. Dik]

Sven Gohlke <sven@clio.in-berlin.de> writes:

>qazmlp wrote:

>>> You amy have used different or dangerous compiler options (like
>>> optimization, never use a '-O') while compiling Your modules.

>>
>> That is the key here...I used '-O' flag. I have been using it for a
>> quite long time. Can you point out any bug in using that flag and the
>> possibilities of that being the cause of the problem that I am facing.

>A library function may expect the return value in a register while the
>caller returns it on the stack. Usual You get this problem only if You
>compile parts with the optimizer and parts without.

Nonsense; any compiler which uses a different calling convention
when specifying any option is broken. PERIOD.

While there are some options to gcc which allow for different calling
conventions, the default has been the system's default calling
convention for some time now.. As long as those options are avoided,
then nothng can go wrong.

When differnet calling conventions are used, the compiler
should generate incompatible objects, such as when using
compiling in 32 or 64 bit mode.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.