Professional Web Applications Themes

Resident set size on Solaris - Sun Solaris

Hi, This is w.r.t Process RSS on Solaris 8. We are running a regression test for one of our processes, i.e. sending the same set of requests, in a mix, continuously. What we are observing is that the RSS of the process is growing continuously whereas the virtual size does not. This we are monitoring with the ps -efly command. What could be the reason for this growth? When/how will it decrease? Is this an ok behaviour? If so, till how long will the RSS keep growing - till it equals the virtual size ?? This also means that the ...

  1. #1

    Default Resident set size on Solaris

    Hi,

    This is w.r.t Process RSS on Solaris 8.
    We are running a regression test for one of our processes,
    i.e. sending the same set of requests, in a mix, continuously.

    What we are observing is that the RSS of the process is growing
    continuously whereas the virtual size does not. This we are
    monitoring with the ps -efly command.
    What could be the reason for this growth?
    When/how will it decrease?
    Is this an ok behaviour?
    If so, till how long will the RSS keep growing - till it
    equals the virtual size ??
    This also means that the system memory usage also keeps
    increasing continuously ?

    I expected that since the same requests are being sent, there is
    no need for this growth to happen - is this assumption correct ?

    Can anyone help me in this regard.

    Thanks & Regards,
    Anil
    Anil Guest

  2. #2

    Default Re: Resident set size on Solaris

    Anil wrote: 

    Try using pmap -x to watch the address space of your process
    during the run.

    The SZ is, I believe, just the total size of the virtual
    memory segments taking no account of whether this memory
    is backed by real physical memory or not.

    If/when we access a page that is not mapped we will
    pagefault to bring that page into physical memory
    (from whereever it lives, or if this is anonymous memory
    we'll arrange an anonymous page). The RSS will increment
    whenever a new physical page is assigned to this process
    address space in this sort of way.
     

    If the process does not undo the mapping/allocation etc
    then the RSS will not shrink unless memory pressure
    (eg, from other applictions) requires that we free
    some pages. If the process releases the pages they'll
    be detached from it's address space and no longer reflect
    in it's RSS.
     

    I can't tell. Probably. But it's possible that the
    appliction is mismanaging it's allocations, and (for
    example) keeps using new buffers instead of reusing
    old ones.
     

    RSS can never exceed the virtual size. It can equal it if we
    touch every page in every segment and nobody forces the
    release of any of these mappings (eg, plenty memory).
    It makes more sense to watch the RSS of each segment
    in the pmap -x output.
     

    RSS on one process could increase while another decreases,
    and system usage would stay the same. Watch the 'free'
    column in vmstat output to see free physical memory.
     

    You'd expect the process to grow to much the same pmap -x
    output (including RSS) if asked to do the same thing
    over and over (assuming no other memory pressure) and
    restarting the process each time.

    Gavin

    Gavin Guest

  3. #3

    Default Re: Resident set size on Solaris

    Anil wrote: 

    I can think of two obvious reasons why this would happen.

    #1: Your process is using so much memory that the system is
    paging heavily. The system will tend to take lots of pages
    away from the process in a big batch, dropping the resident
    size down because pages have been swapped out. Then, as
    your process continues to run, it will cause pages to be
    faulted in, and the resident size will increase gradually.

    If this is the case, it is a problem, and you need to figure
    out a way to make your software use less memory (or buy
    more memory for the systems it runs on). You should be able
    to detect this problem by running "vmstat 5" in a window,
    watching the "sr" column, and then starting the process.
    If the "sr" column jumps up to a larger number, this means
    the system is running low on RAM and is having to search
    for pages it can free.

    #2: Your process has memory mapped a file and is accessing it.
    If you are doing this, then at first the pages from the file
    will just be mapped into the virtual address space but won't
    be using RAM. As you access them, the pages will be brought
    into RAM and your resident size will increase. This is
    not a problem; it's an expected part of how memory-mapped
    files work. The only thing to be aware of is that if you
    access a memory-mapped file that's too big, you could cause
    the system to have to do some work to make sure the right
    pages are in RAM at the right time. (But this still might
    be the best solution.)

    Hope that helps.

    - Logan

    Logan Guest

  4. #4

    Default Re: Resident set size on Solaris

    Hello,

    Thanks for the responses.

    I am looking in the application to see if any changes
    can be made.
    Any tips in this regard?

    Thanks & Regards,
    Anil
    Anil Guest

  5. #5

    Default Re: Resident set size on Solaris


    "Anil" <com> wrote in message
    news:google.com... 

    My two cents:

    * If possible, munmap(2) large, memory mapped files
    when the program is done with them.

    * Try to avoid passing huge data structures to functions
    within the code. Pass pointers instead.

    * Always free(3C) any malloc(3C)-ed buffers as you are
    finished with them. Re-use said buffers if feasable.

    * Cast a beady eye over the code for memory leaks.
     


    --
    Noel R. Nihill
    UNIX® platform development
    Motorola NSS
    I *could* be arguing in my spare time.


    Noel Guest

  6. #6

    Default Re: Resident set size on Solaris


    "Noel R. Nihill" <BLAH.com> wrote in message
    news:bkv3nd$op9$corp.mot.com... 
    >
    > My two cents:
    >
    > * If possible, munmap(2) large, memory mapped files
    > when the program is done with them.
    >
    > * Try to avoid passing huge data structures to functions
    > within the code. Pass pointers instead.
    >
    > * Always free(3C) any malloc(3C)-ed buffers as you are
    > finished with them. Re-use said buffers if feasable.
    >
    > * Cast a beady eye over the code for memory leaks.[/ref]

    Oh. I'll just add that if your app is free of memory leaks
    and you are not mapping enormous files, you just might
    need to get more RAM! Sometimes a cigar is just a cigar...
     [/ref]

    --
    Noel R. Nihill
    UNIX® platform development
    Motorola NSS
    I *could* be arguing in my spare time.


    Noel Guest

  7. #7

    Default Re: Resident set size on Solaris

    In article <bkv457$oqs$corp.mot.com>,
    Noel R. Nihill <BLAH.com> wrote: 

    If the resident size is getting huge, it means you already have enough
    RAM. If you didn't, other processes would be taking RAM away from this
    process and its RSS wouldn't be able to grow so large.

    A large RSS is likely to be an indicator that the application's memory
    access patterns don't exhibit good locality of reference. An example of a
    program like this is "named" (at least through BIND 8, they might have
    improved it in BIND 9); each query that it processes is likely to touch
    many pages in the DNS cache memory that haven't recently been used, and if
    its RSS is significantly lower than its SZ it's probably thrashing and
    performing abysmally.

    If this is the primary application that the system is used for, and it's
    not causing performance problems for other processes on the system, you
    might want to leave well enough alone. But if it looks like the
    application's memory needs are going to grow significantly (which probably
    means that it will need to have more resident memory to maintain the same
    performance), you may want to examine the algorithm to see whether you can
    improve the way it references memory.

    If you process huge arrays or memory-mapped files, try not to loop through
    the whole thing repeatedly; e.g. if the structure of the code is something
    like:

    for each element in array
    do_something(array[i])
    for each element in array
    do_something_else(array[i])

    try changing it to:

    for each element in array
    do_something(array[i])
    do_something_else(array[i])

    --
    Barry Margolin, com
    Level(3), Woburn, MA
    *** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
    Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
    Barry Guest

  8. #8

    Default Re: Resident set size on Solaris

    Noel R. Nihill wrote:
     
     [/ref]
     

    It seems unlikely that it's a memory leak. I believe the
    original poster said that the virtual size remains constant,
    but the resident size is growing. A memory leak would
    cause the virtual size to grow.

    -Logan

    Logan Guest

  9. #9

    Default Re: Resident set size on Solaris


    "Logan Shaw" <rr.com> wrote in message
    news:vHGcb.112741$austin.rr.com... 
    > [/ref]

    >
    > It seems unlikely that it's a memory leak. I believe the
    > original poster said that the virtual size remains constant,
    > but the resident size is growing. A memory leak would
    > cause the virtual size to grow.[/ref]

    Excellent point.
     

    --
    Noel R. Nihill
    UNIX® platform development
    Motorola NSS
    I *could* be arguing in my spare time.


    Noel Guest

  10. #10

    Default Re: Resident set size on Solaris

    "Noel R. Nihill" <BLAH.com> writes in comp.unix.solaris:
    |* If possible, munmap(2) large, memory mapped files
    | when the program is done with them.

    Also look into madvise() if you finish with some regions but not others
    in the mmap()'ed files.

    --
    __________________________________________________ ______________________
    Alan Coopersmith calberkeley.org
    http://www.CSUA.Berkeley.EDU/~alanc/ aka: COM
    Working for, but definitely not speaking for, Sun Microsystems, Inc.
    Alan Guest

Similar Threads

  1. meet your resident mvp ;-)
    By David Bartosik - MS MVP in forum Web Design
    Replies: 6
    Last Post: June 7th, 06:08 PM
  2. [OT] our resident troll
    By Mothra in forum PERL Miscellaneous
    Replies: 3
    Last Post: September 19th, 12:10 PM
  3. recursively finding file size/timestamp on a Mac / Solaris
    By Shishir Saxena in forum PERL Beginners
    Replies: 0
    Last Post: September 17th, 04:07 AM
  4. max IO size, Solaris 8 vxfs QIO, db_multiblock_read_count
    By Sten Rognes in forum Oracle Server
    Replies: 0
    Last Post: December 15th, 04:39 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139