OSX eats unbounded amounts of memory

[fishfry]

I'm trying to understand some virtual memory strangeness I'm seeing.

After a reboot, I have lots of free memory available (as shown by
'top'). After a while, the amount of inactive memory increases until
there's hardly any free memory left, at which point my system starts
getting lots of pageouts.

What is confusing to me is that this condition persists even after I
kill all my applications. Even with no applications running, top
reports a large figure for inactive memory.

It seems to me that when a an application process is killed, its
physical pages should be returned to the free pool, and not left around
as inactive.

Or perhaps that's not what's happening, perhaps some hidden process is
eating all my virtual memory.

The practical result is that no matter how much physical memory I have
in my system, after a certain period of time the system gets into a
state where it's paging out every time I try to do anything. I have to
reboot every day even with a gig of memory installed.

Does anyone have an explanation for why this happens?

[matt neuburg]

In <fishfry-3BB973.08151023072003@netnews.attbi.com> fishfry wrote:

> 'top'). After a while, the amount of inactive memory increases until
> there's hardly any free memory left, at which point my system starts
> getting lots of pageouts.
>
> The practical result is that no matter how much physical memory I have
> in my system, after a certain period of time the system gets into a
> state where it's paging out every time I try to do anything. I have
> to reboot every day even with a gig of memory installed.
>
> Does anyone have an explanation for why this happens?

I don't have an explanation, but you're right, that's the way it works.
The real problem is that as extra swapfiles are generated they too are
not given back when you quit applications, so your hard disk starts
filling up. I wrote my free app MemoryStick exactly so that I could see
memory usage. I restart when I start getting too many pageouts /
swapfiles; I do this several times a day (but I have less RAM than you).
m.

--
matt neuburg, phd = [email]matt@tidbits.com[/email], [url]http://www.tidbits.com/matt[/url]
REALbasic: The Definitive Guide! 2nd edition!
[url]http://www.amazon.com/exec/obidos/ASIN/0596001770/somethingsbymatt[/url]
Subscribe to TidBITS. It's free and smart.

[Steve Bellenot]

In article <fishfry-3BB973.08151023072003@netnews.attbi.com>,
fishfry <fishfry@your-mailbox.com> wrote:

>I'm trying to understand some virtual memory strangeness I'm seeing.
>
>After a reboot, I have lots of free memory available (as shown by
>'top'). After a while, the amount of inactive memory increases until
>there's hardly any free memory left, at which point my system starts
>getting lots of pageouts.
>
>What is confusing to me is that this condition persists even after I
>kill all my applications. Even with no applications running, top
>reports a large figure for inactive memory.
>
>It seems to me that when a an application process is killed, its
>physical pages should be returned to the free pool, and not left around
>as inactive.
>
>Or perhaps that's not what's happening, perhaps some hidden process is
>eating all my virtual memory.
>
>The practical result is that no matter how much physical memory I have
>in my system, after a certain period of time the system gets into a
>state where it's paging out every time I try to do anything. I have to
>reboot every day even with a gig of memory installed.
>
>Does anyone have an explanation for why this happens?

This is normal for a unix system. Generally unix keeps a small
free pool of memory and allocates everything else. Suppose you
use application X and quit it. Rather than work hard to free all the
memory, unix assumes that maybe you will restart application X
and it will be able to reuse the old memory rather than loading
it once again from disk. Actually it makes sense, what application
are you more likely to restart than the one you just quit? (The
oh damn I forgot factor.)

Old versions of unix even had a way of keeping common applications
on the swap partition. It was called the sticky bit.



--
[url]http://www.math.fsu.edu/~bellenot[/url]
bellenot <At/> math.fsu.edu
+1.850.644.7189 (4053fax)

[Tom Harrington]

In article <fishfry-617321.09383523072003@netnews.attbi.com>,
fishfry <fishfry@your-mailbox.com> wrote:

> In article <20030723092014459-0700@news.la.sbcglobal.net>,
> matt neuburg <matt@tidbits.com> wrote:
>

> > In <fishfry-3BB973.08151023072003@netnews.attbi.com> fishfry wrote:
> >

> > > 'top'). After a while, the amount of inactive memory increases until
> > > there's hardly any free memory left, at which point my system starts
> > > getting lots of pageouts.
> > >
> > > The practical result is that no matter how much physical memory I have
> > > in my system, after a certain period of time the system gets into a
> > > state where it's paging out every time I try to do anything. I have
> > > to reboot every day even with a gig of memory installed.
> > >
> > > Does anyone have an explanation for why this happens?

> >
> > I don't have an explanation, but you're right, that's the way it works.
> > The real problem is that as extra swapfiles are generated they too are
> > not given back when you quit applications, so your hard disk starts
> > filling up.

That's not true. I've wondered about this, and I have observed the
number of swap files both increasing and decreasing as memory
requirements vary. Decreasing when, as would be expected, I have quit
applications.

> > I wrote my free app MemoryStick exactly so that I could see
> > memory usage. I restart when I start getting too many pageouts /
> > swapfiles; I do this several times a day (but I have less RAM than you).

> Hi Matt, I keep Memory Stick running all the time, thanks for writing it!
>
> I've been Googling around and apparently there are a lot of reports of
> memory leaks in 10.2.6. See for example
> [url]http://everythingmacintosh.com/news/go.em/Apple/110[/url]. I think this is
> what's going on with my system. My free space gradually disappears
> even before any additional swapfiles are created. By the time the first
> additional swapfile is created it's already time to reboot, because
> freespace is so low that everything I do causes pageouts.

I've seen some such reports as well, most notably at
<http://www.macwrite.com/criticalmass/mac-os-x-memory-leaks.php>. It's
difficult to know where to begin; the article is full of technical
errors and misunderstandings and leads people to all sorts of false
conclusions. The reports tend to have the smell of someone trying to
justify their own existence by inventing problems to warn people of.

> Having to reboot every few hours is totally inconsistent with basing the
> OS on Unix. We'll see if this gets better in future releases.

I have so far not observed this behavior on 10.2.6, which I routinely
run for days or weeks at a time.

--
Tom "Tom" Harrington
Macaroni, Automated System Maintenance for Mac OS X.
Version 1.4: Best cleanup yet, gets files other tools miss.
See [url]http://www.atomicbird.com/[/url]

[Frederick Cheung]

>

> > Having to reboot every few hours is totally inconsistent with basing the
> > OS on Unix. We'll see if this gets better in future releases.

>
> I have so far not observed this behavior on 10.2.6, which I routinely
> run for days or weeks at a time.
>

absolutely, before i rebooted because of the recent security update i
hadn't rebooted in over 3 weeks of constant problem free use.

Fred

[Revanant Morituri]

In article <bfmdl7$4ug$1@news.fsu.edu>,
[email]bellenot@math.fsu.edu[/email] (Steve Bellenot) wrote:

}In article <fishfry-3BB973.08151023072003@netnews.attbi.com>,
}fishfry <fishfry@your-mailbox.com> wrote:
}>I'm trying to understand some virtual memory strangeness I'm seeing.
}>
}>After a reboot, I have lots of free memory available (as shown by
}>'top'). After a while, the amount of inactive memory increases until
}>there's hardly any free memory left, at which point my system starts
}>getting lots of pageouts.
}>
}>What is confusing to me is that this condition persists even after I
}>kill all my applications. Even with no applications running, top
}>reports a large figure for inactive memory.
}>
}>It seems to me that when a an application process is killed, its
}>physical pages should be returned to the free pool, and not left around
}>as inactive.
}>
}>Or perhaps that's not what's happening, perhaps some hidden process is
}>eating all my virtual memory.
}>
}>The practical result is that no matter how much physical memory I have
}>in my system, after a certain period of time the system gets into a
}>state where it's paging out every time I try to do anything. I have to
}>reboot every day even with a gig of memory installed.
}>
}>Does anyone have an explanation for why this happens?
}
}This is normal for a unix system. Generally unix keeps a small
}free pool of memory and allocates everything else. Suppose you
}use application X and quit it. Rather than work hard to free all the
}memory, unix assumes that maybe you will restart application X
}and it will be able to reuse the old memory rather than loading
}it once again from disk. Actually it makes sense, what application
}are you more likely to restart than the one you just quit? (The
}oh damn I forgot factor.)
}
}Old versions of unix even had a way of keeping common applications
}on the swap partition. It was called the sticky bit.

Okay, and with the single digit bus speeds of yesteryear, that sorta
makes sense. The question remains, when does the system decide you
aren't going to relaunch the program, and let something else use the
memory? I only use Photoshop every few days, and it hogs a metric
buttload of space.

--
| /\_/\ | The Genome Diversification Project
|_( ^.^ )_| Because one sentient species on the planet
| >`-'< | is simply too boring for words.
+-Pounce!-+------Remove obvious spamtrap to email------

[Jim Glidewell]

In article <fishfry-3BB973.08151023072003@netnews.attbi.com>, fishfry <fishfry@your-mailbox.com> wrote:

> I'm trying to understand some virtual memory strangeness I'm seeing.
>
> After a reboot, I have lots of free memory available (as shown by
> 'top'). After a while, the amount of inactive memory increases until
> there's hardly any free memory left, at which point my system starts
> getting lots of pageouts.

What method are you using to determine the number of page-outs? Does top
show them?

I see all the behaviors you report *except* pageouts.

> What is confusing to me is that this condition persists even after I
> kill all my applications. Even with no applications running, top
> reports a large figure for inactive memory.

This is per design.

> It seems to me that when a an application process is killed, its
> physical pages should be returned to the free pool, and not left around
> as inactive.

Why, besides "purity of essence"? If the pages are needed, they can be
released instantly (not paged-out) when a new app needs them. (this
applies to "clean" pages - those that still match their on-disk
counterpart)

OTOH, if you re-launch one of the apps you quit, you save a lot of
disk i/o because you don't need to read those pages in again.

OSX's philosophy is "the memory is there, make full use of it." Pages
are retained until something else forces them out. Most of those
inactive pages are effectively disk cache, which is a better use of
memory than leaving it empty.

There are some folks here who claim that the OSX memory management
strategy is "broken", but it seems sensible to me.

> Or perhaps that's not what's happening, perhaps some hidden process is
> eating all my virtual memory.

There are plenty of ways to determine this. "top" is one. Both physical
and virtual memory size is reported for each process.

> The practical result is that no matter how much physical memory I have
> in my system, after a certain period of time the system gets into a
> state where it's paging out every time I try to do anything. I have to
> reboot every day even with a gig of memory installed.

Having memory full is normal, page-outs are not. When this state arises,
run terminal, run "top", and grab/view the data. Page-outs are reported
in the header, both in page-outs since boot and (page-outs in the last
second). Page-ins, BTW, are completely normal and unavoidable.

> Does anyone have an explanation for why this happens?

If you're actually paging-out, then there probably is an errant process.
If you memory is full, things are normal. Your "top" output will tell
you/us a lot.

--
Jim Glidewell
My opinions only

[Tom Harrington]

In article <05e72c217b66f22390c99c68817956ea@TeraNews>,
Revanant Morituri <revanant@obvious.adelphia.net> wrote:

> Okay, and with the single digit bus speeds of yesteryear, that sorta
> makes sense. The question remains, when does the system decide you
> aren't going to relaunch the program, and let something else use the
> memory? I only use Photoshop every few days, and it hogs a metric
> buttload of space.

It lets something else use the memory as soon as the memory is actually
needed for something else. The "inactive" memory of things kept around
just in case is available for use by anything else that might need it.

--
Tom "Tom" Harrington
Macaroni, Automated System Maintenance for Mac OS X.
Version 1.4: Best cleanup yet, gets files other tools miss.
See [url]http://www.atomicbird.com/[/url]

[Gnarlodious]

Yeah, I had that exact same prob. The culprit was an early version of
"System Events" which was unquittable and allocated more and more memory.

-- Gnarlie

[matt neuburg]

In <tph-B18FDC.11114723072003@localhost> Tom Harrington wrote:

> In article <fishfry-617321.09383523072003@netnews.attbi.com>,
> fishfry <fishfry@your-mailbox.com> wrote:
>

>> In article <20030723092014459-0700@news.la.sbcglobal.net>,
>> matt neuburg <matt@tidbits.com> wrote:
>>

>> > I don't have an explanation, but you're right, that's the way it
>> > works. The real problem is that as extra swapfiles are generated
>> > they too are not given back when you quit applications, so your
>> > hard disk starts filling up.

>
> That's not true. I've wondered about this, and I have observed the
> number of swap files both increasing and decreasing as memory
> requirements vary. Decreasing when, as would be expected, I have quit
> applications.

We've had this discussion before, so it's pointless to rehash it. I'm
sure what you're saying is true for you, but what I'm saying is true for
me. I've never seen the number of swapfiles revert of its own accord to,
say, the 1 that is present when I restart; and over time my RAM is
increasingly eaten by Active (not inactive) memory that is not given
back even after I've quit everything I started. This has been true for
the two years I've been monitoring this stuff. (I've had a number of
MemoryStick users write to say that the problem seems worse in recent
versions of the system; I suspect that has something to do with window
caching.) m.

--
matt neuburg, phd = [email]matt@tidbits.com[/email], [url]http://www.tidbits.com/matt[/url]
REALbasic: The Definitive Guide! 2nd edition!
[url]http://www.amazon.com/exec/obidos/ASIN/0596001770/somethingsbymatt[/url]
Subscribe to TidBITS. It's free and smart.

[Wayne C. Morris]

In article <05e72c217b66f22390c99c68817956ea@TeraNews>,
Revanant Morituri <revanant@obvious.adelphia.net> wrote:

> In article <bfmdl7$4ug$1@news.fsu.edu>,
> [email]bellenot@math.fsu.edu[/email] (Steve Bellenot) wrote:
>
> }This is normal for a unix system. Generally unix keeps a small
> }free pool of memory and allocates everything else. Suppose you
> }use application X and quit it. Rather than work hard to free all the
> }memory, unix assumes that maybe you will restart application X
> }and it will be able to reuse the old memory rather than loading
> }it once again from disk. Actually it makes sense, what application
> }are you more likely to restart than the one you just quit? (The
> }oh damn I forgot factor.)
>
> Okay, and with the single digit bus speeds of yesteryear, that sorta
> makes sense. The question remains, when does the system decide you
> aren't going to relaunch the program, and let something else use the
> memory?

When it runs out of "free" memory.

Whenever a program needs some memory, OS X will take it from "free"
memory if possible. If there isn't enough free memory to satisfy the
request, OS X will take it from the "inactive" pile. And if it runs out
of both free and inactive memory, you get lots of pageouts and a
noticeable drop in speed.

In other words, both "free" and "inactive" memory are available for use,
but as long as there's a choice, the "free" memory will be used first.

[fishfry]

In article <jimglidewell-2307031106400001@green.sdc.cs.boeing.com>,
[email]jimglidewell@attbi.com[/email] (Jim Glidewell) wrote:

> In article <fishfry-3BB973.08151023072003@netnews.attbi.com>, fishfry
> <fishfry@your-mailbox.com> wrote:
>

> > I'm trying to understand some virtual memory strangeness I'm seeing.
> >
> > After a reboot, I have lots of free memory available (as shown by
> > 'top'). After a while, the amount of inactive memory increases until
> > there's hardly any free memory left, at which point my system starts
> > getting lots of pageouts.

>
> What method are you using to determine the number of page-outs? Does top
> show them?
>

Yes.

> I see all the behaviors you report *except* pageouts.
>

What OS version are you using? If 10.2.6 that would be interesting.
Apparently some but not all people see this problem

> If you're actually paging-out, then there probably is an errant process.
> If you memory is full, things are normal. Your "top" output will tell
> you/us a lot.

Once I run out of freespace, everything I do results in hundreds of
pageouts as shown by top. Reading mail and loading web pages becomes an
exercize in watching the spinning beachball. Quitting all my active
applications doesn't help. The symptoms are certainly consistent with a
serious memory leak.

[Declan MacLeod]

In article <fishfry-6FD04B.16414023072003@netnews.attbi.com>,
fishfry <fishfry@your-mailbox.com> wrote:

>
>

> > If you're actually paging-out, then there probably is an errant process.
> > If you memory is full, things are normal. Your "top" output will tell
> > you/us a lot.

>
> Once I run out of freespace, everything I do results in hundreds of
> pageouts as shown by top. Reading mail and loading web pages becomes an
> exercize in watching the spinning beachball. Quitting all my active
> applications doesn't help. The symptoms are certainly consistent with a
> serious memory leak.

Have you by chance read this thread?
[url]http://discussions.info.apple.com/WebX?14@63.bntKabLybw1.6@.59972b98[/url]

--
Hey spambots! Harvest this! .!..
Unsolicited commercial email (uce) should be sent to: [email]uce@ftc.gov[/email]