Professional Web Applications Themes

Discrepancy between ps -i -o inblk and figuring numbers by hand - FreeBSD

When I run the command ps -u -o inblock on a process reading a 262144000 byte file the highest block count I see is ~2010 but my filesystem block size is 16384 and 262144000/16384 is 16000 so it seems to be off by about a factor of 8. I even tried looking through the source code for the ps command but my C is not good enough for me to figure it out. Can someone explain or help me figure out what is going on please. Jonathan PS Please CC as I am not currently subscribed to the list. __________________________________ ...

  1. #1

    Default Discrepancy between ps -i -o inblk and figuring numbers by hand

    When I run the command ps -u -o inblock on a process reading a
    262144000 byte file the highest block count I see is ~2010 but my
    filesystem block size is 16384 and 262144000/16384 is 16000 so it seems
    to be off by about a factor of 8. I even tried looking through the
    source code for the ps command but my C is not good enough for me to
    figure it out. Can someone explain or help me figure out what is going
    on please.

    Jonathan

    PS Please CC as I am not currently subscribed to the list.




    __________________________________
    Do you Yahoo!?
    Yahoo! Small Business - Try our new resources site!
    http://smallbusiness./resources/
    Jonathan Guest

  2. #2

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand

    In the last episode (Mar 24), Jonathan Stewart said: 

    That ps column actually looks at the rusage value ru_inblk, which
    doesn't count the number blocks read, but the number of times the
    kernel did physical I/O on behalf of the process (how often the kernel
    blocked doing a read). The clustering code will fetch up to
    sysctl(vfs.read_max) blocks at a time into disk cache, so reading
    sequential files will increment ru_inblock (blocks_in_file/read_max)
    times. Raising vfs.read_max to 32 will greatly increase sequential
    read performance on fast RAID arrays.

    --
    Dan Nelson
    com
    Dan Guest

  3. #3

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand


    --- Dan Nelson <com> wrote: 
    > through 
    > me 
    >
    > That ps column actually looks at the rusage value ru_inblk, which
    > doesn't count the number blocks read, but the number of times the
    > kernel did physical I/O on behalf of the process (how often the
    > kernel
    > blocked doing a read). The clustering code will fetch up to
    > sysctl(vfs.read_max) blocks at a time into disk cache, so reading
    > sequential files will increment ru_inblock (blocks_in_file/read_max)
    > times. Raising vfs.read_max to 32 will greatly increase sequential
    > read performance on fast RAID arrays.
    >[/ref]
    In that case how would I track how much information a process has
    actually read from a drive? I occasionally run processes that will
    read as much as 40+ gig in a single run which takes quite a while and
    on windows :P I can see "bytes read" and "bytes written" per process
    which lets me track how much the program has read so far and thus get
    an idea of how close it is to done. Sorry for the run-on sentence
    there.

    Thanks,
    Jonathan




    __________________________________
    Do you Yahoo!?
    Yahoo! Small Business - Try our new resources site!
    http://smallbusiness./resources/
    Jonathan Guest

  4. #4

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand

    In the last episode (Mar 24), Jonathan Stewart said: 

    I use lsof, which can tell you the file offset of each open
    filedescriptor. "lsof -o -o20 -p ###" will print all the files
    currently opened by pid ###, and their current offset.

    --
    Dan Nelson
    com
    Dan Guest

  5. #5

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand


    --- Dan Nelson <com> wrote: 
    > and 
    > process 
    > get 
    >
    > I use lsof, which can tell you the file offset of each open
    > filedescriptor. "lsof -o -o20 -p ###" will print all the files
    > currently opened by pid ###, and their current offset.
    >[/ref]
    Hmm, that almost works but the program opens 1000's of files each time.
    The program is Unison which is a file synchronizer and I have it
    synchronizing files sets >40GB with and 1000's or more files. Based on
    your description once the file is closed I can't even tell if it was
    read or not :P

    Thanks (a bunch) again,
    Jonathan

    __________________________________________________
    Do You Yahoo!?
    Tired of spam? Yahoo! Mail has the best spam protection around
    http://mail.
    Jonathan Guest

  6. #6

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand

    On 2005-03-24 19:53, Jonathan Stewart <com> wrote: 
    >>
    >> I use lsof, which can tell you the file offset of each open
    >> filedescriptor. "lsof -o -o20 -p ###" will print all the files
    >> currently opened by pid ###, and their current offset.[/ref]
    >
    > Hmm, that almost works but the program opens 1000's of files each
    > time. The program is Unison which is a file synchronizer and I have
    > it synchronizing files sets >40GB with and 1000's or more files.
    > Based on your description once the file is closed I can't even tell if
    > it was read or not :P[/ref]

    So, what you are looking for is a single byte count that increases
    sequentially for all read() and write() system calls?

    Giorgos Guest

  7. #7

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand

    --- GiGiorgoseKeramidaskekeramidaeceidpupatrasr> wrote: [/ref]
    > written" 
    > >
    > > HmHmmthat almost works but the program opens 1000's of files each
    > > time. The program is Unison which is a file synchronizer and I[/ref]
    > have 
    > if 
    >
    > So, what you are looking for is a single byte count that increases
    > sequentially for all read() and write() system calls?
    >[/ref]
    Pretty much, yes. To be specific all read() and write() calls for a
    given process. Even something that counted in 512 byte or UFUFSlocks
    would be useful.

    Thanks,
    Jonathan



    __________________________________
    Do you Yahoo!?
    Yahoo! Mail - 250MB free storage. Do more. Manage less.
    http://info.mail./mail_250
    Jonathan Guest

  8. #8

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand

    On 2005-03-25 10:08, Jonathan Stewart <com> wrote: 
    >
    > Pretty much, yes. To be specific all read() and write() calls for a
    > given process. Even something that counted in 512 byte or UFUFSlocks
    > would be useful.[/ref]

    To what end, may I ask? Per process statistics may include byte counts from a
    few thousand threads that read and/or write from a few hundred descriptors.

    Even per file descriptor statistics quickly get useless when one considers
    that a single byte read may cause the read-ahead of a few thousand bytes or
    that a single write may reach the corresponding device several seconds later.

    Giorgos Guest

  9. #9

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand


    --- Giorgos Keramidas <upatras.gr> wrote: [/ref]
    > increases 
    > >
    > > Pretty much, yes. To be specific all read() and write() calls for a
    > > given process. Even something that counted in 512 byte or[/ref]
    > UFUFSlocks 
    >
    > To what end, may I ask? Per process statistics may include byte
    > counts from a
    > few thousand threads that read and/or write from a few hundred
    > descriptors.
    >
    > Even per file descriptor statistics quickly get useless when one
    > considers
    > that a single byte read may cause the read-ahead of a few thousand
    > bytes or
    > that a single write may reach the corresponding device several
    > seconds later.
    >[/ref]
    As I mentioned in an earlier email my main use of this is really just
    for one program. I can do a du to find out how much information it
    needs to read and then by watching how much it has read get a rough
    idea of how much longer it will be. Not really a necessary feature
    just a "nice to have" kind of thing.

    Jonathan



    __________________________________
    Do you Yahoo!?
    Yahoo! Small Business - Try our new resources site!
    http://smallbusiness./resources/
    Jonathan Guest

  10. #10

    Default Re: Discrepancy between ps -i -o inblk and figuring numbers by hand

    On 2005-03-25 20:17, Jonathan Stewart <com> wrote: 
    >>
    >> To what end, may I ask? Per process statistics may include byte
    >> counts from a
    >> few thousand threads that read and/or write from a few hundred
    >> descriptors.
    >>
    >> Even per file descriptor statistics quickly get useless when one
    >> considers
    >> that a single byte read may cause the read-ahead of a few thousand
    >> bytes or
    >> that a single write may reach the corresponding device several
    >> seconds later.[/ref]
    >
    > As I mentioned in an earlier email my main use of this is really just
    > for one program. I can do a du to find out how much information it
    > needs to read and then by watching how much it has read get a rough
    > idea of how much longer it will be. Not really a necessary feature
    > just a "nice to have" kind of thing.[/ref]

    I can try hacking something together, but the main difficulty of making
    modifications to the struct filedesc of each file descriptor is that
    userlevel programs need some way of getting access to this information
    too, i.e. through an ioctl() on the descriptor.

    Giorgos Guest

Similar Threads

  1. Q: Problem figuring out versions of DBI
    By David in forum PERL Miscellaneous
    Replies: 14
    Last Post: October 17th, 04:07 AM
  2. need help figuring out how to delete rows?
    By Deadsam in forum PHP Development
    Replies: 3
    Last Post: August 25th, 06:03 AM
  3. Help with figuring out password
    By Alan Ames in forum Macromedia Dreamweaver
    Replies: 1
    Last Post: July 9th, 07:45 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139