set noexec_user_stack=1 in /etc/system

[Joerg Schilling]

While this works nice on a sparc based system, it does not seem to
work on Solaris x86.

This has been noticed while trying to convert a Linux based Web server to
Solaris.

--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
[email]js@cs.tu-berlin.de[/email] (uni) If you don't have iso-8859-1
[email]schilling@fokus.fraunhofer.de[/email] (work) chars I am J"org Schilling
URL: [url]http://www.fokus.fraunhofer.de/usr/schilling[/url] [url]ftp://ftp.berlios.de/pub/schily[/url]

[Casper H.S. Dik]

[email]js@cs.tu-berlin.de[/email] (Joerg Schilling) writes:

>While this works nice on a sparc based system, it does not seem to
>work on Solaris x86.

>This has been noticed while trying to convert a Linux based Web server to
>Solaris.

That's because the Intel CPU can't distinguish between "read" and
"execute" permissions of individual pages.

You could tinker with segment permissions but then the issue is that
mprotect() no longer works on those pages.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

[Oscar del Rio]

Joerg Schilling wrote:

> While this works nice on a sparc based system, it does not seem to
> work on Solaris x86.

From the Solaris Tunable Parameters Reference Manual:

A Solaris system running a 64-bit kernel makes the stacks of all 64-bit
applications non-executable by default. Setting this variable is
necessary to make 32-bit applications non-executable on systems running
64-bit or 32-bit kernels.

Note –

This variable exists on all systems running the Solaris 2.6, 7, 8, or 9
releases, but it is only effective on sun4u and sun4m architectures.