Professional Web Applications Themes

Available memory - UNIX Programming

Is there a function call, or a set of calls, that will tell me how much physical memory is available to grab with malloc? I'm sure there is, but I can't find it....

  1. #1

    Default Available memory

    Is there a function call, or a set of calls, that will tell me how much
    physical memory is available to grab with malloc?

    I'm sure there is, but I can't find it.
    Allo Guest

  2. #2

    Default Re: Available memory



    Allo wrote:
     

    see if there's an getrlimit(2) function in your
    distribution.

    man 2 getrlimit

    It's a Berkleyism as far as I can tell but may
    be in some other kernels as well.

    --

    "It is impossible to make anything foolproof because fools are so
    ingenious" - A. Bloch

    Nick Guest

  3. #3

    Default Re: Available memory

    In article <hR80c.113749$ops.worldnet.att.net>,
    att.net says... 
    > =20 
    >
    > see if there's an getrlimit(2) function in your
    > distribution.
    >
    > man 2 getrlimit
    >
    > It's a Berkleyism as far as I can tell but may
    > be in some other kernels as well.[/ref]

    Yeah, there is. (I'm using Solaris, by the way.)

    And it just occurred to me that I could always look at the source code
    for "top" and see how that prog checks for quantities of available
    memory. Should've thought of that sooner.
    Allo Guest

  4. #4

    Default Re: Available memory

    On Sat, 28 Feb 2004, Allo wrote:
     

    sysconf is a good one - but the amountof physical memory you have
    and what's available for use by malloc two entirely different beasts.
    All memory allocated to your process comes from a pool of virtual
    memory.

    --
    Rich Teer, SCNA, SCSA

    President,
    Rite Online Inc.

    Voice: +1 (250) 979-1638
    URL: http://www.rite-online.net
    Rich Guest

  5. #5

    Default Re: Available memory

    in comp.unix.programmer i read:
     

    why would you want to do that -- just allocate what you need. other
    programs may need some memory, and if you are hogging it all they may fail.

    --
    a signature
    those Guest

  6. #6

    Default Re: Available memory


    "those who know me have no need of my name" <net>
    wrote in message news:net... 
     [/ref]


    The 'malloc' function allocates virtual memory, not physical memory, so
    your question makes no sense. A typical UNIX machine has a physical memory
    availability target (usually 2-32Mb) and there is nearly always that amount
    free.

    Free memory is only good for certain very specific things, such as
    handling a sudden burst of incoming network traffic. Other than that, free
    memory is wasted memory and there is no good reason for an operating system
    to waste anything.
     


    Or ask the real question rather than how to implement a solution that
    won't work. ;)

    DS




    David Guest

  7. #7

    Default Re: Available memory

    In article <c1roo3$o75$webmaster.com>, com
    says... 
    > [/ref]
    >
    >
    > The 'malloc' function allocates virtual memory, not physical memory, so
    > your question makes no sense.[/ref]

    Physical memory is part of virtual memory, so malloc actually does
    allocate physical memory. If I determine the amount of unused physical
    memory, then a request for a percentage of that amount should result in
    an allocation that resides entirely within physical RAM. This should be
    true unless the amount of allocated virtual memory exceeds the machine's
    physical memory capacity (adjusted for whatever amount of physical
    memory has been reserved by the operating system).
     

    If free memory is wasted memory, then a "physical memory availability
    target" (i.e., an amount of physical memory that's always free) means
    that memory waste is actually part of the system design.
    Allo Guest

  8. #8

    Default Re: Available memory



    Allo wrote: 
    >> 
    >>
    >>
    >> The 'malloc' function allocates virtual memory, not physical memory, so
    >>your question makes no sense.[/ref]
    >
    >
    > Physical memory is part of virtual memory, so malloc actually does
    > allocate physical memory. If I determine the amount of unused physical
    > memory, then a request for a percentage of that amount should result in
    > an allocation that resides entirely within physical RAM. This should be
    > true unless the amount of allocated virtual memory exceeds the machine's
    > physical memory capacity (adjusted for whatever amount of physical
    > memory has been reserved by the operating system).
    >
    >  
    >
    >
    > If free memory is wasted memory, then a "physical memory availability
    > target" (i.e., an amount of physical memory that's always free) means
    > that memory waste is actually part of the system design.[/ref]

    Actually, you're both right.

    In the "usual world" of *nix, you often exceed physical
    memory as processes get created, live for a few seconds
    or a few minuts, then go away. (Transients).
    Every command typed into a shell command line
    is like this. Paging portions of processes to
    th swap device is how the O/S takes care of this
    condition. In theory, if I had only one process,
    it could grow to physical memory size plus
    swap partition size. (I said in theory).

    This implies that in order to run a process
    once it is next in the pecking order, some
    or all of the process image has to be
    read in from disk again. On systems with
    strict response time requirements this
    is considered a "BAD THING" because it leads
    to unpredictable response times. At least
    even more unpredictable than just because
    of the scheduling algorithm.

    Those kinds of systems (and I have worked on
    several of the over the years) generally
    have long lived processes to do the majority
    of the work (a few maintenance tasks are
    transients but carefully controlled), and
    the size of each process is also carefully
    controlled by the design.

    e.g. Total Memory: 2 GB
    Database memory: 1,570 MB
    O/S & Utilities: 100 MB
    Communications Subsystem: 60 MB
    Application Logic Handlers: 150 MB
    Reserve: 120 MB (space for transients in here)

    Our statisics indicate that we generally
    page on average of only once or twice per second
    and most of this happens during the short time
    period when transients pop up to monitor the
    health and welfare of the syste and to
    gather statistics on the operation.

    --

    "It is impossible to make anything foolproof because fools are so
    ingenious" - A. Bloch

    Nick Guest

  9. #9

    Default Re: Available memory

    Allo wrote:
     
    >
    > Physical memory is part of virtual memory, so malloc actually does
    > allocate physical memory. If I determine the amount of unused
    > physical memory, then a request for a percentage of that amount should
    > result in an allocation that resides entirely within physical RAM.[/ref]

    Actually, it might result in no real allocation at all. Many kernels do
    lazy allocation, which means that the actual memory allocation doesn't
    happen before you access the memory. Therefore, it can even happen that
    you allocate more memory than is physically or virtually available.
     
    >
    > If free memory is wasted memory, then a "physical memory availability
    > target" (i.e., an amount of physical memory that's always free) means
    > that memory waste is actually part of the system design.[/ref]

    Under some systems, you often have only few memory free, because the
    system tries to use memory for buffers and caches instead of letting it
    stay unused. However, as soon as a program needs that memory, it's made
    available for it.

    Rolf Guest

  10. #10

    Default Re: Available memory

    > The 'malloc' function allocates virtual memory, not physical 

    Most modern UNIX systems are virtual memory systems. When you request a
    large allocation, sometimes you get physical RAM; sometimes you get swap
    space; sometimes both. Also, sometimes you get nothing... by which I mean,
    the kernel delays allocating anything until you actually start using the
    area. Here's a description of copy-on-write, used during fork and calloc
    http://en.wikipedia.org/wiki/Copy-on-write

    This all makes it very difficult to determine what exactly is available (in
    terms of resources).

    --
    Jem Berkes
    http://www.sysdesign.ca/
    Jem Guest

  11. #11

    Default Re: Available memory

    On Sat, 28 Feb 2004 17:38:16 -0500
    Allo <s> wrote:
     
    To add to what has already been discussed, it is very implementation
    specific. Depends if you are on a 32 or 64 bit system, BSD or SysV, and
    how the OS manages its own memory. Most Unix systems historically used
    brk(2) and sbrk(2) to grow the heap, and when it cannot grow any
    more,that becomes your limit unless you have already his the artificial
    limit. But, newer mallocs may use mmap(2) to allocate pages as they are
    needed. This is more flexible since mmap(2) may allocate pages anywhere
    in the process' virtual memory space where sbrk(2) can only allocate in
    a contiguous space. And, as I mentioned, memory management is very
    implementation specific. Other than that, there are tools (like
    getrlimit()) that can give you some information to make a guess.

    --
    Jerry Feldman <gaf-nospam-at-blu.org>
    Boston Linux and Unix user group
    http://www.blu.org PGP key id:C5061EA9
    PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
    Jerry Guest

  12. #12

    Default Re: Available memory

    Jerry Feldman <org> writes:
     
    > To add to what has already been discussed, it is very implementation
    > specific.[/ref]

    And anyway, you don't grab physical memory, you grab virtual memory:
    the OS can swap your process out anytime.

    Perhaps there are some platform specific extensions allowing usercode
    to wire a page, but the more wired pages there are the more easy it is
    to trash the system with swaps, to crash the processes if the system
    does overcommiting, or even to panic it with resource exhaustion.
    Niceties we all need to have some fun in our dull lifes...

    --
    __Pascal_Bourguignon__ http://www.informatimago.com/
    There is no worse tyranny than to force a man to pay for what he doesn't
    want merely because you think it would be good for him.--Robert Heinlein
    http://www.theadvocates.org/
    Pascal Guest

Similar Threads

  1. Out-of-Memory Error - Acrobat Professional7.0.8 - But there is plenty of memory?
    By Clifford_Scott@adobeforums.com in forum Adobe Acrobat Macintosh
    Replies: 2
    Last Post: September 26th, 12:18 PM
  2. Flash Player 8 huge memory using during drag (all memory gone in few minutes) !!!
    By LSopko@1024Informatica.com in forum Macromedia Flash Player
    Replies: 0
    Last Post: June 13th, 09:42 AM
  3. Memory consumption of Ruby/mod_ruby combo on Apache [memory leak]
    By David Heinemeier Hansson in forum Ruby
    Replies: 4
    Last Post: September 10th, 01:58 AM
  4. Replies: 1
    Last Post: June 26th, 04:00 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139