2 GB Memory Limit

[Helge Preuss]

Hi all,
i read that Linux enables a process to use 3 GB of user space memory.
But on my (redhat 8.0, 2.4.18-14bigmem) box it seems impossible to use
more than 2 GB. Have I forgot anything to set up my system to use the
full 3 GB?
Any help appreciated
Helge

PS demo program:

#include <iostream>

using namespace std;

int main () {
int ** dummy = new int * [8];
// this loop should run min. 5, maybe 6 times, but it doesn't:
for (int i = 0; i < 8; i++) {
cout << i+1 << "*512 MB ";
dummy[i] = (int *)malloc (1024*1024*512); // alloc 512 MB
if (dummy[i]) {
dummy[i][1024*1024*127] = 0; // access mem
cout << "ok" << endl; // success
}
else {
cout << "failed!" << endl;
return 0;
}
}
return 0;
}


Its output is:
1*512 MB ok
2*512 MB ok
3*512 MB ok
4*512 MB ok
5*512 MB failed!

[Helge Preuss]

Hi all,
i read that Linux enables a process to use 3 GB of user space memory.
But on my (redhat 8.0, 2.4.18-14bigmem) box it seems impossible to use
more than 2 GB. Have I forgot anything to set up my system to use the
full 3 GB?
Any help appreciated
Helge

PS demo program:

#include <iostream>

using namespace std;

int main () {
int ** dummy = new int * [8];
// this loop should run min. 5, maybe 6 times, but it doesn't:
for (int i = 0; i < 8; i++) {
cout << i+1 << "*512 MB ";
dummy[i] = (int *)malloc (1024*1024*512); // alloc 512 MB
if (dummy[i]) {
dummy[i][1024*1024*127] = 0; // access mem
cout << "ok" << endl; // success
}
else {
cout << "failed!" << endl;
return 0;
}
}
return 0;
}


Its output is:
1*512 MB ok
2*512 MB ok
3*512 MB ok
4*512 MB ok
5*512 MB failed!

[Peter T. Breuer]

Helge Preuss <spam.preuss@fhi-berlin.mpg.de> wrote:

> i read that Linux enables a process to use 3 GB of user space memory.
> But on my (redhat 8.0, 2.4.18-14bigmem) box it seems impossible to use
> more than 2 GB. Have I forgot anything to set up my system to use the
> full 3 GB?

Yes, you have. Recompile your kernel appropriately. I believe a 1+3
split is an option.

Peter

[Helge Preuss]

Peter T. Breuer wrote:

> Helge Preuss <spam.preuss@fhi-berlin.mpg.de> wrote:
>

>>i read that Linux enables a process to use 3 GB of user space memory.
>>But on my (redhat 8.0, 2.4.18-14bigmem) box it seems impossible to use
>>more than 2 GB. Have I forgot anything to set up my system to use the
>>full 3 GB?

>
>
> Yes, you have. Recompile your kernel appropriately. I believe a 1+3
> split is an option.
>
> Peter

Can you point me to the configuration option? I did not find it in any
of the categories which I think are appropriate (Processor type &
features, General setup - and some more which don't look appropriate)
TIA
Helge

[David]

Helge Preuss wrote:

> Peter T. Breuer wrote:
>

>> Helge Preuss <spam.preuss@fhi-berlin.mpg.de> wrote:
>>

>>> i read that Linux enables a process to use 3 GB of user space memory.
>>> But on my (redhat 8.0, 2.4.18-14bigmem) box it seems impossible to
>>> use more than 2 GB. Have I forgot anything to set up my system to use
>>> the full 3 GB?

>>
>>
>>
>> Yes, you have. Recompile your kernel appropriately. I believe a 1+3
>> split is an option.
>>
>> Peter

>
>
> Can you point me to the configuration option? I did not find it in any
> of the categories which I think are appropriate (Processor type &
> features, General setup - and some more which don't look appropriate)
> TIA
> Helge
>

# Processor type and features
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y <--- Upto 4GB memory
# CONFIG_HIGHMEM64G is not set <--- more than 4GB memory
CONFIG_HIGHMEM=y
CONFIG_HIGHIO=y

--
Confucius: He who play in root, eventually kill tree.
Registered with The Linux Counter. [url]http://counter.li.org/[/url]
Slackware 9.0 Kernel 2.4.22 i686 (GCC) 3.3
Uptime: 6 days, 13:49, 1 user, load average: 0.10, 0.22, 0.20

[Helge Preuss]

> # Processor type and features

> # CONFIG_NOHIGHMEM is not set
> CONFIG_HIGHMEM4G=y <--- Upto 4GB memory
> # CONFIG_HIGHMEM64G is not set <--- more than 4GB memory
> CONFIG_HIGHMEM=y
> CONFIG_HIGHIO=y

I don't want to play stupid, but: are you sure this works? I'd like to
be certain before I reboot the system, because it is already in use. The
documentation is not explicit about CONFIG_HIGHMEM - it just says that
CONFIG_NOHIGHMEM results in a 3GB/1GB split. CONFIG_HIGHMEM is not
documented at all, only CONFIG_HIGHMEM4G/64G. It's not documented how
the memory is divided with those. I use CONFIG_HIGHMEM64G. But apart
from CONFIG_HIGHMEM=y, which is missing on my box, it seems the
configuration is exactly the same on my system. So what does
CONFIG_HIGHMEM=y do? Or, for that matter, CONFIG_HIGHMEM64G?

Helge

[Peter T. Breuer]

Helge Preuss <spam.preuss@fhi-berlin.mpg.de> wrote:

> Peter T. Breuer wrote:

>> Helge Preuss <spam.preuss@fhi-berlin.mpg.de> wrote:
>>

>>>i read that Linux enables a process to use 3 GB of user space memory.
>>>But on my (redhat 8.0, 2.4.18-14bigmem) box it seems impossible to use
>>>more than 2 GB. Have I forgot anything to set up my system to use the
>>>full 3 GB?

>>
>>
>> Yes, you have. Recompile your kernel appropriately. I believe a 1+3
>> split is an option.

> Can you point me to the configuration option? I did not find it in any

Not unless I go look. Do you want me to? All I will do is grep through
Configure.help for "memory".

nbd:/usr/oboe/ptb% grep -i memory /usr/local/src/linux-2.4.20-xfs/Documentation/Configure.help
marketed by the Digital Equipment Corporation of blessed memory, now
High Memory support
Linux can use up to 64 Gigabytes of physical memory on x86 systems.
physical memory, not all of it can be "permanently mapped" by the
kernel. The physical memory that's not permanently mapped is called
"high memory".
split: 3GB are mapped so that each process sees a 3GB virtual memory
space and the remaining part of the 4GB virtual memory space is used
by the kernel to permanently map as much physical memory as
The actual amount of total physical memory will either be auto
...

There you are. Look for that. "split: 3GB".

> of the categories which I think are appropriate (Processor type &
> features, General setup - and some more which don't look appropriate)

Peter

[Helge Preuss]

> nbd:/usr/oboe/ptb% grep -i memory /usr/local/src/linux-2.4.20-xfs/Documentation/Configure.help
> marketed by the Digital Equipment Corporation of blessed memory, now

Nice one :-)

> Linux can use up to 64 Gigabytes of physical memory on x86 systems.
> physical memory, not all of it can be "permanently mapped" by the
> kernel. The physical memory that's not permanently mapped is called
> "high memory".
> split: 3GB are mapped so that each process sees a 3GB virtual memory
> space and the remaining part of the 4GB virtual memory space is used
> by the kernel to permanently map as much physical memory as
> The actual amount of total physical memory will either be auto

Found that already. To ask you the same question as I asked David: can
you confirm it really works for you? I.e., did you ever successfully
allocate >2 GB in one process? The docs are ambiguous on that topic.
Helge

[David]

Helge Preuss wrote:

>
> I don't want to play stupid, but: are you sure this works? I'd like to
> be certain before I reboot the system, because it is already in use. The
> documentation is not explicit about CONFIG_HIGHMEM - it just says that
> CONFIG_NOHIGHMEM results in a 3GB/1GB split. CONFIG_HIGHMEM is not
> documented at all, only CONFIG_HIGHMEM4G/64G. It's not documented how
> the memory is divided with those. I use CONFIG_HIGHMEM64G. But apart
> from CONFIG_HIGHMEM=y, which is missing on my box, it seems the
> configuration is exactly the same on my system. So what does
> CONFIG_HIGHMEM=y do? Or, for that matter, CONFIG_HIGHMEM64G?

I don't have any systems with more than 2GB of RAM so I haven't
had any need to test any larger RAM configurations.

--
Confucius: He who play in root, eventually kill tree.
Registered with The Linux Counter. [url]http://counter.li.org/[/url]
Slackware 9.0 Kernel 2.4.22 i686 (GCC) 3.3
Uptime: 6 days, 16:19, 1 user, load average: 0.42, 0.32, 0.22

[TCS]

On Fri, 05 Sep 2003 19:19:40 +0200, Helge Preuss <spam.preuss@fhi-berlin.mpg.de> wrote:

>

>> nbd:/usr/oboe/ptb% grep -i memory /usr/local/src/linux-2.4.20-xfs/Documentation/Configure.help
>> marketed by the Digital Equipment Corporation of blessed memory, now

> Nice one :-)

>> Linux can use up to 64 Gigabytes of physical memory on x86 systems.
>> physical memory, not all of it can be "permanently mapped" by the
>> kernel. The physical memory that's not permanently mapped is called
>> "high memory".
>> split: 3GB are mapped so that each process sees a 3GB virtual memory
>> space and the remaining part of the 4GB virtual memory space is used
>> by the kernel to permanently map as much physical memory as
>> The actual amount of total physical memory will either be auto

> Found that already. To ask you the same question as I asked David: can
> you confirm it really works for you? I.e., did you ever successfully
> allocate >2 GB in one process? The docs are ambiguous on that topic.

IIRC, support for >4GB is only for the total system.
You'll never address more than 4GB on a processor with 32 bit addressing
such as the intel x86 familly. Less (like 2GB) if the high address bit
is attached meaning like "system vs. user".

[Helge Preuss]

> IIRC, support for >4GB is only for the total system.

> You'll never address more than 4GB on a processor with 32 bit addressing
> such as the intel x86 familly. Less (like 2GB) if the high address bit
> is attached meaning like "system vs. user".

That's correct. I just want to maximize the amount of memory the user
processes get. And i had the impression, reading previous postings and
the kernel docs, that it is possible to divide the memory 3GB:1GB, (even
a patch enabling a 3.5:0.5 GB split was mentioned). Now it seems that
the 3GB:1GB split is only possible for machines with <1GB of RAM, which
seems ridiculous to me. But it seems I have to live with it.