Professional Web Applications Themes

Post encodeWithCoder stall? (archiving large graphs part 3) - Mac Programming

In order to save large and tangled graphs without getting recursive unarchiving I decided to take the graph apart and archive it in the form of a series of linear arrays. What I have now are two linear arrays holding the elements of the graph. The elements themselves are archived without any connection to other elements. I also have three linear arrays where each object is a small array with NSNumber instances. These three arrays show how the graph is to be reassembled when loading. As before, a small graph saves and loads fine (a hundred elements or so). Increase ...

  1. #1

    Default Post encodeWithCoder stall? (archiving large graphs part 3)

    In order to save large and tangled graphs without getting recursive
    unarchiving I decided to take the graph apart and archive it in the
    form of a series of linear arrays. What I have now are two linear
    arrays holding the elements of the graph. The elements themselves are
    archived without any connection to other elements. I also have three
    linear arrays where each object is a small array with NSNumber
    instances. These three arrays show how the graph is to be reassembled
    when loading.

    As before, a small graph saves and loads fine (a hundred elements or
    so). Increase the size and I get a few seconds delay when the root
    object's encodeWithCoder is done ( a few hundred elements). Increase
    the size more (a thousand or so), and I get to watch the spinning
    beach-ball for a few minute (other applications are not affected). The
    resulting file was only 850 kb and loaded the way it was supposed to
    (fast and no stalling).
    I've increased the size to over 12000 elements, and the beach-ball
    has been spinning for over one and a half hour, and is still spinning
    (going to kill it when I've posted this). The NSLog() comments tell me
    encodeWithCoder is done. There is no new file on my HD, and according
    to top, my app is using about 70% of my CPU.

    Any idea why encodeWithCoder would stall like that when it is
    finished? It's called from NSDoent, btw.

    --
    C Lund, www.notam02.no/~clund
    C Guest

  2. #2

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    C Lund wrote: 

    What does Sampler or Shark say it's spending its time? One possible
    bottleneck might be NSArchiver determining if each object has been
    already archived. Since it sounds like you've done that work already, a
    custom NSCoder class might get you better performance. Also, keyed
    archiving is slower than linear archiving.

    -Petre

    Peter Guest

  3. #3

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    In article <cd4mms$12u$apple.com>,
    Peter Ammon <com> wrote:
     
    >
    > What does Sampler or Shark say it's spending its time?[/ref]

    Sampler or Shark? I'm not familiar with them.
     

    I'll give non-keyed archiving a shot. But I don't see why I'm getting
    this huge stall when encodeWithCoder is done.

    The more I tangle with it, the worse NSArchiver looks. Seems it can't
    handle complex graphs, and now, not even large arrays. Writing a
    custom archiver class is becoming more and more attractive.
     

    --
    C Lund, www.notam02.no/~clund
    C Guest

  4. #4

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    In article <cd4mms$12u$apple.com>,
    Peter Ammon <com> wrote:
     

    I looked around and found Sampler on my HD, and I've downloaded the
    other apps now - I assume Shark is among them.
     

    I'm not familiar enough with Sampler to read it's output properly, but
    from what I can see, the app is just busy archiving the data. However,
    I don't understand why it should take five minutes or more just to
    archive a 800 kb file..
     

    --
    C Lund, www.notam02.no/~clund
    C Guest

  5. #5

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    In article <cd4mms$12u$apple.com>,
    Peter Ammon <com> wrote:
     

    Turns out this was the solution. Going from NSKeyedArchiving to
    NSArchiving got rid of the entire "post encodeWithCoder stall". B)

    Can't help but wonder what NSKeyedArchiving was doing with all that
    time..

    But I can't help but notice that unarchiving is a lot faster than
    archiving. A 2.3 MB file pretty much plops right in when I load it,
    but it took maybe 10-20 seconds seconds to save. I can live with that
    though. B)
     

    --
    C Lund, www.notam02.no/~clund
    C Guest

  6. #6

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    In article <chello.com>,
    C Lund <no> wrote:
     

    Again, without seeing the code it is nearly impossible to tell. My
    guess based on seeing the old code is that your change in archiving
    (i.e., addition of many NSNumber arrays) creates a ton of temporary
    objects that are getting released in the next run loop.
    Doc Guest

  7. #7

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    On Thu, 15 Jul 2004, C Lund wrote:
     
    >
    > Turns out this was the solution. Going from NSKeyedArchiving to
    > NSArchiving got rid of the entire "post encodeWithCoder stall". B)
    >
    > Can't help but wonder what NSKeyedArchiving was doing with all that
    > time..[/ref]

    NSKeyedArchiver has a "feature" (which I consider to be a design flaw)
    where it uniques every array that's put into the archive. That is to say,
    if you archive two arrays that contain exactly the same objects,
    NSKeyedArchiver will only archive one copy of the array to save space. As
    you might imagine, checking for duplicates is really slow. In fact, it
    will be O(n^2) on the number of arrays you archive. All of this uniquing
    happens after you've finished archiving, which is why everything seems to
    go very fast and then stall at the end. You said you were archiving a lot
    of small arrays, so this is almost certainly your problem. If you really
    want to use keyed archiving, reducing or eliminating your usage of arrays
    (and anything that gets archived like arrays, such as sets and
    dictionaries) will probably help a great deal.
     

    NSCoding is inteded primarily for saving and loading nibs. For nibs, you
    create them once and load them millions of times, so it's more important
    for loading to be fast than saving to be fast. You're probably seeing the
    results of that optimization.
    Michael Guest

  8. #8

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    In article
    <supernews.com 
    Doc O'Leary <example.com> wrote:
     
    >
    > Again, without seeing the code it is nearly impossible to tell. My
    > guess based on seeing the old code is that your change in archiving
    > (i.e., addition of many NSNumber arrays) creates a ton of temporary
    > objects that are getting released in the next run loop.[/ref]

    The main cause turned out to be that I was using KeyedArchiver instead
    of Archiver. But there's still a tiny stall there. Not big enough to
    be a problem though.

    However, is the creation / release of temporary objects generally
    considered a costly thing wrt to time?

    --
    C Lund, www.notam02.no/~clund
    C Guest

  9. #9

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    In article <twistedsys.net>,
    Michael Ash <com> wrote: 
    > > Turns out this was the solution. Going from NSKeyedArchiving to
    > > NSArchiving got rid of the entire "post encodeWithCoder stall". B)
    > > Can't help but wonder what NSKeyedArchiving was doing with all that
    > > time..[/ref]
    > NSKeyedArchiver has a "feature" (which I consider to be a design flaw)
    > where it uniques every array that's put into the archive. That is to say,
    > if you archive two arrays that contain exactly the same objects,
    > NSKeyedArchiver will only archive one copy of the array to save space. As
    > you might imagine, checking for duplicates is really slow. In fact, it
    > will be O(n^2) on the number of arrays you archive.[/ref]

    That explains it..
     

    I've dumped the keying. The only reason I used it was to deal with the
    problem I had the first time we talked about this. B)

    I hope Apple does more work on the archiving and coding classes. Seems
    to me they're a bit limited in capability.

    --
    C Lund, www.notam02.no/~clund
    C Guest

  10. #10

    Default Re: Post encodeWithCoder stall? (archiving large graphs part 3)

    On Fri, 16 Jul 2004, C Lund wrote:
     

    As with so many things, the answer is "it depends". Whenever your program
    is performing badly, the first thing you should do is run Shark on it and
    see what it's doing. If it's spending a lot of time creating and
    destroying temporary objects, then the answer is "yes".
    Michael Guest

Similar Threads

  1. Large scale website design - Part 2
    By JakeFlynn in forum Macromedia ColdFusion
    Replies: 8
    Last Post: March 10th, 06:32 PM
  2. Archiving large graphs (part II)
    By C in forum Mac Programming
    Replies: 26
    Last Post: June 29th, 08:39 AM
  3. Archiving large graphs
    By C Lund in forum Mac Programming
    Replies: 22
    Last Post: October 3rd, 02:32 PM
  4. Replies: 0
    Last Post: July 9th, 05:10 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139