Archiving large graphs

[C Lund]

I have a class Root with contains two NSMutableArrays *aList, *bList.
These arrays point to all instances of ClassA and ClassB. It is
encoded by calling archiveRootObject: toFile:

-(void)encodeWithCoder:(NSCoder *)coder
{
[coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&aList];
[coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&bList];
}

ClassA has it's own NSMutableArray *bList and is archived by:

-(void)encodeWithCoder:(NSCoder *)coder
{
[coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&bList];
}

ClassA's array points only to some instances of ClassB.

ClassB has NSMutableArrays *aList, *bList, which only point to some
instances of ClassA and ClassB, and is archived by:

-(void)encodeWithCoder:(NSCoder *)coder
{
[coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&aList];
[coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&bList];
}

This setup works fine as long as there aren't too many instances of
ClassA and ClassB, but when the numbers get too high (around a
thousand instances), the app crashes with a signal 11 (SIGSEGV) when I
try to archive the file.

According to the documentation, the above causes circular references
which would cause crashes (but why does it work when the number of
instances of ClassA and ClassB are low?), so I tried archiving them
this way instead:

[coder encodeValueOfObjCType:@encode(int) at:&aListSize];
for (i=0;i<aListSize;i++)
[coder encodeObject:[aList objectAtIndex:i]];
[coder encodeValueOfObjCType:@encode(int) at:&bListSize];
for (i=0;i<bListSize;i++)
[coder encodeObject:[bList objectAtIndex:i]];

But this still crashes when aList and bList get too large - also, I
can't get the smaller files to load when saved this way. I tried doing
it by:

[coder decodeValueOfObjCType:@encode(int) at:&aListSize];
for (i=0;i<aListSize;i++)
[aList addObject:[[coder decodeObject] retain]];
[coder decodeValueOfObjCType:@encode(int) at:&bListSize];
for (i=0;i<bListSize;i++)
[bList addObject:[[coder decodeObject] retain]];

Not that that matters that much since I can save large files anyway.

Ideas?

--
C Lund, www.notam02.no/~clund

[Per Bull Holmen]

C Lund <clund@NOSPAMnotam02.no> wrote:

> I have a class Root with contains two NSMutableArrays *aList, *bList.
> These arrays point to all instances of ClassA and ClassB. It is
> encoded by calling archiveRootObject: toFile:
>
> -(void)encodeWithCoder:(NSCoder *)coder
> {
> [coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&aList];
> [coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&bList];
> }

Why don't you just encode the arrays as objects?

[coder encodeObject:aList];
[coder encodeObject:bList];

etc...

I don't know if it will make any difference, but that's how I would do
it.

Per

[Michael Ash]

In article <clund-D1FE69.12090327092003@amstwist00.chello.com>,
C Lund <clund@NOSPAMnotam02.no> wrote:

> I have a class Root with contains two NSMutableArrays *aList, *bList.
> These arrays point to all instances of ClassA and ClassB. It is
> encoded by calling archiveRootObject: toFile:
>
> -(void)encodeWithCoder:(NSCoder *)coder
> {
> [coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&aList];
> [coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&bList];
> }
>
> ClassA has it's own NSMutableArray *bList and is archived by:
>
> -(void)encodeWithCoder:(NSCoder *)coder
> {
> [coder encodeValueOfObjCType:@encode(NSMutableArray *) at:&bList];
> }

If you want to encode an object, use encodeObject:. Using
encodeValueOfObjCType: for an object is sort of stupid. It probably ends
up working the exact same way, but encodeObject: is a lot clearer and
easier to type. Who knows, maybe it'll even fix your problem.

Have you tried running this thing in a debugger and seeing where it
crashes? If you haven't, you really should, otherwise you're just
shooting in the dark.

Without a backtrace and goodies like that, I can only guess at the cause
of your problem. However, I had a very similar problem, so I'm guessing
that it's the same cause. Circular references are explicitly allowed by
NSCoder. The class knows how to handle them and this absolutely will not
cause a problem of any kind. The problem is that object graphs are
encoded recursively.

Imagine you have object A which then does [coder encodeObject:B], and B
does encodeObject:C, and so on. This ends up calling encodeWithCoder:
and encodeObject: in turn, recursively. If your object graph is large
enough and shaped right to cause the recursion to go on long enough, you
run out of stack space and crash. This could very well be considered a
bug with NSCoder, since it's not really a *requirement* to execute these
methods recursively, and doing so causes various weird problems like
this, but that's how it works now so we have to live with it. If this is
the problem, it'll be instantly recognizable in the debugger, because
first it will take a long time for the backtrace to show up, and when it
finally does, it'll be several hundred or thousand functions deep.

> According to the documentation, the above causes circular references
> which would cause crashes (but why does it work when the number of
> instances of ClassA and ClassB are low?), so I tried archiving them
> this way instead:
>
> [coder encodeValueOfObjCType:@encode(int) at:&aListSize];
> for (i=0;i<aListSize;i++)
> [coder encodeObject:[aList objectAtIndex:i]];
> [coder encodeValueOfObjCType:@encode(int) at:&bListSize];
> for (i=0;i<bListSize;i++)
> [coder encodeObject:[bList objectAtIndex:i]];
>
> But this still crashes when aList and bList get too large

This is just a very complicated way of saying [coder
encodeObject:aList]; [coder encodeObject:bList];. It functions in
exactly the same way and has the exact same problems.

However, you can use this code but with encodeConditionalObject:, which
won't trigger the recursion, and then in your Root class you encode the
entire list of B's all at once. This will get all the references right
without actually doing the encoding recursively. It's ugly but it should
work.

> - also, I
> can't get the smaller files to load when saved this way.

I'm not sure why this would be. You'll have to spend some time with the
debugger.

> I tried doing
> it by:
>
> [coder decodeValueOfObjCType:@encode(int) at:&aListSize];
> for (i=0;i<aListSize;i++)
> [aList addObject:[[coder decodeObject] retain]];
> [coder decodeValueOfObjCType:@encode(int) at:&bListSize];
> for (i=0;i<bListSize;i++)
> [bList addObject:[[coder decodeObject] retain]];

These retains cause a memory leak, they aren't necessary and shouldn't
be there.

Also, this is not really related to your problem, but if you don't need
to run on 10.1, consider using NSKeyedCoding. This makes life quite a
bit easier in various ways, such as removing the requirements that you
encode and decode values in exactly the same order, and that you can't
skip over decoding or encoding value. You'll still have the same
problems with deep recursion, though.

I hope all this is clear....

[C Lund]

In article <1g1y3x8.15yvvbe1l63ybkN%pbh_news@yahoo.com>,
[email]pbh_news@yahoo.com[/email] (Per Bull Holmen) wrote:

> Why don't you just encode the arrays as objects?

> [coder encodeObject:aList];
> [coder encodeObject:bList];

> etc...

And then decode them like this

aList = [coder decodeObject];
bList = [coder decodeObject];

you mean?

> I don't know if it will make any difference, but that's how I would do
> it.

I've tried that. It doesn't work at all. I can save, but any attempt
at loading gives me a signal 10 (SIGSEGV) crash.

> Per

--
C Lund, www.notam02.no/~clund

[C Lund]

I wrote a longer reply to this, but then MT-NW locked up right before
I was about to send the thing. Bp

In article <mail-29AD7D.13155027092003@localhost>,
Michael Ash <mail@mikeash.com> wrote:

> If you want to encode an object, use encodeObject:. Using
> encodeValueOfObjCType: for an object is sort of stupid. It probably ends
> up working the exact same way, but encodeObject: is a lot clearer and
> easier to type. Who knows, maybe it'll even fix your problem.

It didn't (assuming I did it correctly); see my reply to Holmen.

> Have you tried running this thing in a debugger and seeing where it
> crashes? If you haven't, you really should, otherwise you're just
> shooting in the dark.
>
> Without a backtrace and goodies like that, I can only guess at the cause
> of your problem. However, I had a very similar problem, so I'm guessing
> that it's the same cause. Circular references are explicitly allowed by
> NSCoder. The class knows how to handle them and this absolutely will not
> cause a problem of any kind. The problem is that object graphs are
> encoded recursively.
>
> Imagine you have object A which then does [coder encodeObject:B], and B
> does encodeObject:C, and so on. This ends up calling encodeWithCoder:
> and encodeObject: in turn, recursively. If your object graph is large
> enough and shaped right to cause the recursion to go on long enough, you
> run out of stack space and crash.

That sounds about right.

> This could very well be considered a
> bug with NSCoder, since it's not really a *requirement* to execute these
> methods recursively, and doing so causes various weird problems like
> this, but that's how it works now so we have to live with it. If this is
> the problem, it'll be instantly recognizable in the debugger, because
> first it will take a long time for the backtrace to show up, and when it
> finally does, it'll be several hundred or thousand functions deep.

That's what I suspected. The actual number of instances wasn't that
important. I found that making thousands of instances that were in
seperate bunches of only a few hundred instances each didn't cause any
trouble, even ifthe total count was in the thousands. But if I linked
those bunches together, the thing crashes.

> > According to the documentation, the above causes circular references
> > which would cause crashes (but why does it work when the number of
> > instances of ClassA and ClassB are low?), so I tried archiving them
> > this way instead:
> >
> > [coder encodeValueOfObjCType:@encode(int) at:&aListSize];
> > for (i=0;i<aListSize;i++)
> > [coder encodeObject:[aList objectAtIndex:i]];
> > [coder encodeValueOfObjCType:@encode(int) at:&bListSize];
> > for (i=0;i<bListSize;i++)
> > [coder encodeObject:[bList objectAtIndex:i]];
> >
> > But this still crashes when aList and bList get too large

>
> This is just a very complicated way of saying [coder
> encodeObject:aList]; [coder encodeObject:bList];. It functions in
> exactly the same way and has the exact same problems.
>
> However, you can use this code but with encodeConditionalObject:, which
> won't trigger the recursion, and then in your Root class you encode the
> entire list of B's all at once. This will get all the references right
> without actually doing the encoding recursively. It's ugly but it should
> work.

So encodeConditionalObject will only encode the single object without
ferreting out every little object in the same network? Sounds like
it's worth a try.

> > - also, I
> > can't get the smaller files to load when saved this way.

> I'm not sure why this would be. You'll have to spend some time with the
> debugger.

> > I tried doing
> > it by:
> >
> > [coder decodeValueOfObjCType:@encode(int) at:&aListSize];
> > for (i=0;i<aListSize;i++)
> > [aList addObject:[[coder decodeObject] retain]];
> > [coder decodeValueOfObjCType:@encode(int) at:&bListSize];
> > for (i=0;i<bListSize;i++)
> > [bList addObject:[[coder decodeObject] retain]];

>
> These retains cause a memory leak, they aren't necessary and shouldn't
> be there.

I put the retains there because the above code seemed to produce empty
arrays (which again caused the crashes - I think). They made no
difference one way or the other so I left them.

> Also, this is not really related to your problem, but if you don't need
> to run on 10.1,

Well, I'm running 10.1.5, so it is probably necessary.. ;)

> consider using NSKeyedCoding. This makes life quite a
> bit easier in various ways, such as removing the requirements that you
> encode and decode values in exactly the same order, and that you can't
> skip over decoding or encoding value. You'll still have the same
> problems with deep recursion, though.

That sounds like it could be useful at some later time.

> I hope all this is clear....

Well, we'll see. Hopefully encodeConditionalObject will fix the
problem.

--
C Lund, www.notam02.no/~clund

[Per Bull Holmen]

C Lund <clund@NOSPAMnotam02.no> wrote:

> In article <1g1y3x8.15yvvbe1l63ybkN%pbh_news@yahoo.com>,
> [email]pbh_news@yahoo.com[/email] (Per Bull Holmen) wrote:
>

> > Why don't you just encode the arrays as objects?

>

> > [coder encodeObject:aList];
> > [coder encodeObject:bList];

>

> > etc...

>
> And then decode them like this
>
> aList = [coder decodeObject];
> bList = [coder decodeObject];
>
> you mean?

Yeah, but you gotta retain them unless they're only temporary.

> > I don't know if it will make any difference, but that's how I would do
> > it.

>
> I've tried that. It doesn't work at all. I can save, but any attempt
> at loading gives me a signal 10 (SIGSEGV) crash.

OK, but the error is somewhere else, the method for coding/decoding the
arrays are correct.

SIGSEGV errors can happen for many reasons. One reason could be that you
don't retain the arrays, and later try to access them. You must use the
debugger in PB and run a bracktrace after it has crashed, bt.

Per

[C Lund]

In article <1g1ylny.93gzg1cvccmkN%pbh_news@yahoo.com>,
[email]pbh_news@yahoo.com[/email] (Per Bull Holmen) wrote:

> > And then decode them like this
> >
> > aList = [coder decodeObject];
> > bList = [coder decodeObject];
> >
> > you mean?

>
> Yeah, but you gotta retain them unless they're only temporary.

Yeah, that fixed one problem. B)

--
C Lund, www.notam02.no/~clund

[C Lund]

As things are now, the Root class is encoded like this

[coder encodeObject:aList];
[coder encodeObject:bList];

and decoded like this

aList = [[coder decodeObject] retain];
bList = [[coder decodeObject] retain];

---

ClassA's encoding:

[coder encodeValueOfObjCType:@encode(int) at:&bListSize];
for (i=0;i<bListSize;i++)
[coder encodeConditionalObject:[bList objectAtIndex:i]];

and decoding:

bList = [[NSMutableArray alloc] init];
[coder decodeValueOfObjCType:@encode(int) at:&bListSize];
for (i=0;i<bListSize;i++)
[bList addObject:[coder decodeObject]];

---

ClassB's encoding:

for (i=0;i<3;i++)
[coder encodeConditionalObject:[aList objectAtIndex:i]];
[coder encodeValueOfObjCType:@encode(int) at:&bListSize];
for (i=0;i<bListSize;i++)
[coder encodeConditionalObject:[bList objectAtIndex:i]];

and decoding:

aList = [[NSMutableArray alloc] init];
for (i=0;i<3;i++)
[aList addObject:[coder decodeObject]];
[coder decodeValueOfObjCType:@encode(int) at:&bListSize];
bList = [[NSMutableArray alloc] init];
for (i=0;i<bListSize;i++)
[bList addObject:[coder decodeObject]];

---

This works, but no better than the code in my first post in this
thread. IOW it still crashes when encoding if the number of instances
is in the thousands. Guess it's time to start with the debugger.

--
C Lund, www.notam02.no/~clund

[Michael Ash]

In article <clund-5BA38C.19201027092003@amstwist00.chello.com>,
C Lund <clund@NOSPAMnotam02.no> wrote:

> So encodeConditionalObject will only encode the single object without
> ferreting out every little object in the same network? Sounds like
> it's worth a try.

The function of encodeObject: is clear. It more or less directly calls
[object encodeWithCoder:...], and so on.

EncodeConditionalObject: is used for things where you may not always
want to encode an object. Think of a Thing class, which is part of a
BigCollectionOfThings. You may want to archive a collection, or just a
single Thing. If you archive the collection, each Thing needs a pointer
back to its collection, but if you just archive a Thing, you don't want
to drag the entire collection into the archive. Then you use
encodeConditionalObject:. What it does is, if somebody *else* uses
encodeObject: on the same object, it gets encoded like usual. If nobody
else does, then nothing gets encoded, and when you later decodeObject,
you get a nil.

The effect here is, basically, to delay encoding. You get a reference,
but you don't trigger the actual encoding until somebody else makes it
happen, so you can hopefully avoid the recursion.

[from your other post...]

> This works, but no better than the code in my first post in this
> thread. IOW it still crashes when encoding if the number of instances
> is in the thousands. Guess it's time to start with the debugger.

I don't see anything obviously wrong with the code you posted. You need
to find out where it's crashing and why, so you can find out where the
recursion is and figure out how to stop it. Really, the time to start
with the debugger is not now, it was when you started having the
problem. But better late than never. The debugger will tell you exactly
what you need to know. I normally run my programs in the debugger out of
habit, so I can see what happened during a crash. If for whatever reason
I wasn't in the debugger and a crash happened, the first thing I do is
try to duplicate the crash while running in the debugger so I can see
what happened.

[C Lund]

In article <mail-DB20BD.01133228092003@localhost>,
Michael Ash <mail@mikeash.com> wrote:

> In article <clund-5BA38C.19201027092003@amstwist00.chello.com>,
> C Lund <clund@NOSPAMnotam02.no> wrote:
> EncodeConditionalObject: is used for things where you may not always
> want to encode an object. Think of a Thing class, which is part of a
> BigCollectionOfThings. You may want to archive a collection, or just a
> single Thing. If you archive the collection, each Thing needs a pointer
> back to its collection, but if you just archive a Thing, you don't want
> to drag the entire collection into the archive. Then you use
> encodeConditionalObject:. What it does is, if somebody *else* uses
> encodeObject: on the same object, it gets encoded like usual. If nobody
> else does, then nothing gets encoded, and when you later decodeObject,
> you get a nil.

> The effect here is, basically, to delay encoding. You get a reference,
> but you don't trigger the actual encoding until somebody else makes it
> happen, so you can hopefully avoid the recursion.

What makes it happen?

> [from your other post...]

> > This works, but no better than the code in my first post in this
> > thread. IOW it still crashes when encoding if the number of instances
> > is in the thousands. Guess it's time to start with the debugger.

>
> I don't see anything obviously wrong with the code you posted. You need
> to find out where it's crashing and why, so you can find out where the
> recursion is and figure out how to stop it.

Well, I put a few NSLogs in there. The app runs fine on the first pass
of

[coder encodeObject:aList];
[coder encodeObject:bList];

But on the second pass things go haywire. The recursion seems to take
place in the second pass of the aList encoding - because all of a
sudden I'm getting a lot of bList entites in the mix, and the
instances never complete their encoding but rather pass on to the next
instance.

> Really, the time to start
> with the debugger is not now, it was when you started having the
> problem.

I know. But the thought of stepping through literally thousands of
breakpoints isn't very tempting. I'm generally a bit shakey with the
thing. Is there a command that tells you the depth of a given
recursion - such as the one taking place during the encoding of the
lists?

> But better late than never. The debugger will tell you exactly
> what you need to know. I normally run my programs in the debugger out of
> habit, so I can see what happened during a crash. If for whatever reason
> I wasn't in the debugger and a crash happened, the first thing I do is
> try to duplicate the crash while running in the debugger so I can see
> what happened.

I've done that, but I'm not familiar enough with the debugger to know
what to look for. I get an "EXC_BAD_ACCESS" message when the app
crashes. What does that mean?

--
C Lund, www.notam02.no/~clund

[Michael Ash]

In article <clund-E05985.18154029092003@amstwist00.chello.com>,
C Lund <clund@NOSPAMnotam02.no> wrote:

> In article <mail-DB20BD.01133228092003@localhost>,
> Michael Ash <mail@mikeash.com> wrote:
>

> > The effect here is, basically, to delay encoding. You get a reference,
> > but you don't trigger the actual encoding until somebody else makes it
> > happen, so you can hopefully avoid the recursion.

>
> What makes it happen?

I'm not privy to the design details of NSCoder, but I assume it's
something like this:

Somebody says [coder encodeConditionalObject:Q]. The coder first checks
to see if Q has already been encoded, and if it has makes this a
reference to Q as usual. Otherwise, it marks this down as a reference to
Q, but doesn't actually encode Q. Then, once encoding has finished, it
goes through all conditional references and sees if they were encoded
later. If they were, it hooks them up, and if they weren't it sets them
to nil.

> Well, I put a few NSLogs in there. The app runs fine on the first pass
> of
>
> [coder encodeObject:aList];
> [coder encodeObject:bList];
>
> But on the second pass things go haywire. The recursion seems to take
> place in the second pass of the aList encoding - because all of a
> sudden I'm getting a lot of bList entites in the mix, and the
> instances never complete their encoding but rather pass on to the next
> instance.

Are the instances using encodeObject: or encodeConditionalObject:? If
they aren't encoding conditionally, then they'll encode the entire
subgraph they're connected to. But if they are conditional, you have to
make sure that somebody else encodes them unconditionally.

> > Really, the time to start
> > with the debugger is not now, it was when you started having the
> > problem.

>
> I know. But the thought of stepping through literally thousands of
> breakpoints isn't very tempting. I'm generally a bit shakey with the
> thing. Is there a command that tells you the depth of a given
> recursion - such as the one taking place during the encoding of the
> lists?

The debugger will automatically break when your program crashes. You
won't need breakpoints, at least not right away. Just run your program
in the debugger, and make it crash. The debugger will then show you
exactly where the crash was, what the chain of calling functions was,
and the values of variables at that point. PB's debugger is just a
wrapper around gdb, so you can look at information about gdb if you
don't know how to do something. Gdb is weird and difficult but powerful.

Note that if you're crashing from too much recursion, it may take a
while for the debugger to finish building the function call stack after
the program crashes, since it's kind of slow.

> > But better late than never. The debugger will tell you exactly
> > what you need to know. I normally run my programs in the debugger out of
> > habit, so I can see what happened during a crash. If for whatever reason
> > I wasn't in the debugger and a crash happened, the first thing I do is
> > try to duplicate the crash while running in the debugger so I can see
> > what happened.

>
> I've done that, but I'm not familiar enough with the debugger to know
> what to look for. I get an "EXC_BAD_ACCESS" message when the app
> crashes. What does that mean?

It means that your program tried to access memory that it was not
allowed to access. This can happen for lots of reasons, such as
following a pointer to a deallocated object or (in this case, we assume)
overflowing your stack. The debugger will show you exactly where the bad
access is happening, although this is often *not* the error, but merely
a delayed consequence of it.

[C Lund]

In article <mail-D0DF2F.18421329092003@localhost>,
Michael Ash <mail@mikeash.com> wrote:

> > Well, I put a few NSLogs in there. The app runs fine on the first pass
> > of
> >
> > [coder encodeObject:aList];
> > [coder encodeObject:bList];

I'm beginning to wonder whether maybe I should make these conditional
too, and then build the lists the same way as I do in ClassA and
ClassB. Can't hurt to try.

> > But on the second pass things go haywire. The recursion seems to take
> > place in the second pass of the aList encoding - because all of a
> > sudden I'm getting a lot of bList entites in the mix, and the
> > instances never complete their encoding but rather pass on to the next
> > instance.

> Are the instances using encodeObject: or encodeConditionalObject:?

encodeConditionalObject:

ClassA's encoding:

[coder encodeValueOfObjCType:@encode(int) at:&bListSize];
for (i=0;i<bListSize;i++)
[coder encodeConditionalObject:[bList objectAtIndex:i]];

ClassB's encoding:

for (i=0;i<3;i++)
[coder encodeConditionalObject:[aList objectAtIndex:i]];
[coder encodeValueOfObjCType:@encode(int) at:&bListSize];
for (i=0;i<bListSize;i++)
[coder encodeConditionalObject:[bList objectAtIndex:i]];

> If
> they aren't encoding conditionally, then they'll encode the entire
> subgraph they're connected to. But if they are conditional, you have to
> make sure that somebody else encodes them unconditionally.

You lost me there. Who encodes them unconditionally? How?

> > I know. But the thought of stepping through literally thousands of
> > breakpoints isn't very tempting. I'm generally a bit shakey with the
> > thing. Is there a command that tells you the depth of a given
> > recursion - such as the one taking place during the encoding of the
> > lists?

> The debugger will automatically break when your program crashes. You
> won't need breakpoints, at least not right away. Just run your program
> in the debugger, and make it crash. The debugger will then show you
> exactly where the crash was, what the chain of calling functions was,
> and the values of variables at that point. PB's debugger is just a
> wrapper around gdb, so you can look at information about gdb if you
> don't know how to do something. Gdb is weird and difficult but powerful.
>
> Note that if you're crashing from too much recursion, it may take a
> while for the debugger to finish building the function call stack after
> the program crashes, since it's kind of slow.

Well, the call stack (at least I think that's what this is) comes out
like this:

Thread-1,
0 _encodePointerIfNew
1 _encodeObject
2 -[NSArchiver encodeObject:]
3 -[NSArchiver encodeConditionalObject:]
4 -[ClassA encodeWithCoder:]

(lots of repetitions with ClassA and ClassB encodings as above in no
particular order snipped)

And these are the last things to happen before the recursion gets out
of hand:

500 -[ClassB encodWithCoder:]
500 _encodeObject
500 -[NSArchiver encodeObject:]
500 -[NSArchiver encodeConditionalObject:]
500 -[ClassA encodeWithCoder:]
500 _encodeObject
501 -[NSArchiver encodeObject:]
501 -[NSCoder encodeByCopyObject:]
501 -[Array encodeWithCoder:]
501 _encodeObject
501 -[NSArchiver encodeObject:]
501 -[RootClass encodeWithCoder:]
501 _encodeObject
501 -[NSArchiver encodeObject:]
501 -[NSArchiver encodeRootObject:]
501 +[NSArchiver archiveDataWithRootObject:]
502 +[NSArchiver archiveRootObject:toFile:]
502 -[ControllerClass didEnd:returnCode:contextInfo:]
(more stuff snipped)

Thread-3
0 syscall_thread_switch
1 +[NSThread sleepUntilDate:]
2 -[NSUIHeartBeat_heartBeatThread:]
3 forThreadForFunction
4 _pthread_body

Thread-4
0 mach_msg_overwrite_trap
1 mach_msg
2 _pthread_become_available
3 pthread_exit
4 _pthread_body

This doesn't tell me more than that the app crashes due to a recursion
that never ends - which I already knew.

What are these "Threads"? Where is "Thread-2"?

> > > But better late than never. The debugger will tell you exactly
> > > what you need to know. I normally run my programs in the debugger out of
> > > habit, so I can see what happened during a crash. If for whatever reason
> > > I wasn't in the debugger and a crash happened, the first thing I do is
> > > try to duplicate the crash while running in the debugger so I can see
> > > what happened.

> > I've done that, but I'm not familiar enough with the debugger to know
> > what to look for. I get an "EXC_BAD_ACCESS" message when the app
> > crashes. What does that mean?

> It means that your program tried to access memory that it was not
> allowed to access. This can happen for lots of reasons, such as
> following a pointer to a deallocated object or (in this case, we assume)
> overflowing your stack. The debugger will show you exactly where the bad
> access is happening, although this is often *not* the error, but merely
> a delayed consequence of it.

Such as the stack overflowing because of a runaway recursion, yes?

--
C Lund, www.notam02.no/~clund

[Michael Ash]

In article <clund-BD1152.16330430092003@amstwist00.chello.com>,
C Lund <clund@NOSPAMnotam02.no> wrote:

> In article <mail-D0DF2F.18421329092003@localhost>,
> Michael Ash <mail@mikeash.com> wrote:
>

> > > Well, I put a few NSLogs in there. The app runs fine on the first pass
> > > of
> > >
> > > [coder encodeObject:aList];
> > > [coder encodeObject:bList];

>
> I'm beginning to wonder whether maybe I should make these conditional
> too, and then build the lists the same way as I do in ClassA and
> ClassB. Can't hurt to try.
>

> > If
> > they aren't encoding conditionally, then they'll encode the entire
> > subgraph they're connected to. But if they are conditional, you have to
> > make sure that somebody else encodes them unconditionally.

>
> You lost me there. Who encodes them unconditionally? How?

The whole point of encodeConditionalObject: is to encode a pointer to an
object that you may not need. You basically register your interest in
that object, but don't actually make it get encoded. If somebody *else*
does encodeObject: on the same object (this is what I mean by
"unconditionally") then the object gets encoded, and your conditional
encode gets hooked up. Otherwise you get a nil.

Where you normally use this is a situation like this. Let's say you have
a World class and a Person class. A World instance contains a list of
Person instances, and a Person instance contains a pointer back to the
World. Now you may want to encode an entire World, or you may want to
encode a single Person by itself. In this case, in Person, you do [coder
encodeConditionalObject:world]. That way, if the World is already being
encoded, you get hooked up, and if it's not, you don't make it get
encoded just for your single Person instance.

But we're using it to keep things from becoming recursive. The strategy
is to have a graph of objects and a list of every object in the graph in
parallel. In the graph, you use encodeConditionalObject: to encode
pointers but not do recursive encoding. Then we encodeObject: the list
of objects to make sure every object really gets encoded.

> Well, the call stack (at least I think that's what this is) comes out
> like this:
>
> Thread-1,
> 0 _encodePointerIfNew
> 1 _encodeObject
> 2 -[NSArchiver encodeObject:]
> 3 -[NSArchiver encodeConditionalObject:]
> 4 -[ClassA encodeWithCoder:]
>
> (lots of repetitions with ClassA and ClassB encodings as above in no
> particular order snipped)
>
> And these are the last things to happen before the recursion gets out
> of hand:
>
> 500 -[ClassB encodWithCoder:]
> 500 _encodeObject
> 500 -[NSArchiver encodeObject:]
> 500 -[NSArchiver encodeConditionalObject:]
> 500 -[ClassA encodeWithCoder:]
> 500 _encodeObject
> 501 -[NSArchiver encodeObject:]
> 501 -[NSCoder encodeByCopyObject:]
> 501 -[Array encodeWithCoder:]
> 501 _encodeObject
> 501 -[NSArchiver encodeObject:]
> 501 -[RootClass encodeWithCoder:]
> 501 _encodeObject
> 501 -[NSArchiver encodeObject:]
> 501 -[NSArchiver encodeRootObject:]
> 501 +[NSArchiver archiveDataWithRootObject:]
> 502 +[NSArchiver archiveRootObject:toFile:]
> 502 -[ControllerClass didEnd:returnCode:contextInfo:]
> (more stuff snipped)

You actually got these backwards; the higher numbers are lower on the
stack, and the last things to happen before the crash are what you have
up at the top. But it's not that important, except that you have to
remember that each function was called by the function below it in the
list, not the one above.

> Thread-3
> 0 syscall_thread_switch
> 1 +[NSThread sleepUntilDate:]
> 2 -[NSUIHeartBeat_heartBeatThread:]
> 3 forThreadForFunction
> 4 _pthread_body
>
> Thread-4
> 0 mach_msg_overwrite_trap
> 1 mach_msg
> 2 _pthread_become_available
> 3 pthread_exit
> 4 _pthread_body
>
> This doesn't tell me more than that the app crashes due to a recursion
> that never ends - which I already knew.

Actually, it tells you a lot of valiable information. I think you
snipped a little too much, but that's ok. (And apparently you were
typing this out by hand? Try typing "bt" in the Console tab, then
copy/paste the output if you need to send another call stack.)
Basically, from here you can see exactly where the recursion is
occuring, although it doesn't necessary look like anything new. It looks
like, as we thought, that A encodes B encodes A repetitively; but this
is somehow happening through encodeConditionalObject:. If you click on a
function in the call stack and the function is one of yours, the
debugger will jump to the source for that function and let you examine
the state of all the variables, and see exactly where you were at the
time. That might help.

> What are these "Threads"? Where is "Thread-2"?

Threads is a whole other topic of discussion. Basically, you can have
multiple points of execution in a single program. If you didn't create
any threads, you probably don't need to worry about them; these extra
threads were created by the system frameworks and they're normal. I
assume Thread-2 existed and subsequently exited.

> > It means that your program tried to access memory that it was not
> > allowed to access. This can happen for lots of reasons, such as
> > following a pointer to a deallocated object or (in this case, we assume)
> > overflowing your stack. The debugger will show you exactly where the bad
> > access is happening, although this is often *not* the error, but merely
> > a delayed consequence of it.

>
> Such as the stack overflowing because of a runaway recursion, yes?

Well, a stack overflow causes a crash right away, so that's not what I
meant by a "delayed consequence", but that will result in a segfault or
other such error such as you saw. Here's a very quick example of the
kind of delayed action you can have:

void func(void)
{
int array[4];
int *x;
int y;

x = &y;

for(y = 0; y <= 4; y++) // should be <, not <=
array[y] = y; // this goes up to array[4], not good

*x = 0; // this will probably crash
}

The error here is in the for loop's conditional, which allows y to go up
to 4 before exiting. (The declaration array[4] only allows us to use
indices 0 through 3). When we access index 4, we're doing something
illegal, but it probably won't crash; C compilers don't usually
range-check. Instead, it merrily overwrites whatever's in the spot where
index 4 would be, which in this case is probably x. So x now contains
'4' instead of a pointer to y, and when we access x we'll crash.

This gets really fun when you have the error that overwrites data in
function X, and then the overwritten data is accessed ten minutes later
in function Y.

Anyway, this is a big digression, since your problem is fortunately a
lot more straightforward than that. :)

[C Lund]

In article <mail-FD96BE.20532630092003@localhost>,
Michael Ash <mail@mikeash.com> wrote:

> In article <clund-BD1152.16330430092003@amstwist00.chello.com>,
> C Lund <clund@NOSPAMnotam02.no> wrote:

> > > If
> > > they aren't encoding conditionally, then they'll encode the entire
> > > subgraph they're connected to. But if they are conditional, you have to
> > > make sure that somebody else encodes them unconditionally.

> > You lost me there. Who encodes them unconditionally? How?

> The whole point of encodeConditionalObject: is to encode a pointer to an
> object that you may not need. You basically register your interest in
> that object, but don't actually make it get encoded. If somebody *else*
> does encodeObject: on the same object (this is what I mean by
> "unconditionally") then the object gets encoded, and your conditional
> encode gets hooked up. Otherwise you get a nil.

Ok. I just got confused when you started talking about "somebody
encoding [Š] unconditionally". ;)

(person & world example)

> But we're using it to keep things from becoming recursive. The strategy
> is to have a graph of objects and a list of every object in the graph in
> parallel. In the graph, you use encodeConditionalObject: to encode
> pointers but not do recursive encoding. Then we encodeObject: the list
> of objects to make sure every object really gets encoded.

And as far as I can see, that's what my code *should* be doing, yet
that's not what's happeing.

I'm supposed to be using encodeWithCoder:, right? There aren't any
other encoding methods I should know of?

> > Well, the call stack (at least I think that's what this is) comes out
> > like this:

> > Thread-1,
> > 0 _encodePointerIfNew
> > 1 _encodeObject
> > 2 -[NSArchiver encodeObject:]
> > 3 -[NSArchiver encodeConditionalObject:]
> > 4 -[ClassA encodeWithCoder:]

> > (lots of repetitions with ClassA and ClassB encodings as above in no
> > particular order snipped)

> > And these are the last things to happen before the recursion gets out
> > of hand:

> > 500 -[ClassB encodWithCoder:]
> > 500 _encodeObject
> > 500 -[NSArchiver encodeObject:]
> > 500 -[NSArchiver encodeConditionalObject:]
> > 500 -[ClassA encodeWithCoder:]
> > 500 _encodeObject
> > 501 -[NSArchiver encodeObject:]
> > 501 -[NSCoder encodeByCopyObject:]
> > 501 -[Array encodeWithCoder:]
> > 501 _encodeObject
> > 501 -[NSArchiver encodeObject:]
> > 501 -[RootClass encodeWithCoder:]
> > 501 _encodeObject
> > 501 -[NSArchiver encodeObject:]
> > 501 -[NSArchiver encodeRootObject:]
> > 501 +[NSArchiver archiveDataWithRootObject:]
> > 502 +[NSArchiver archiveRootObject:toFile:]
> > 502 -[ControllerClass didEnd:returnCode:contextInfo:]
> > (more stuff snipped)

>
> You actually got these backwards; the higher numbers are lower on the
> stack, and the last things to happen before the crash are what you have
> up at the top.

I know; what I meant was that the 500's were the last things that took
place before the *recursion* started. The crash happened after the
_encodePointerIfNew. B)

> > Thread-3
> > 0 syscall_thread_switch
> > 1 +[NSThread sleepUntilDate:]
> > 2 -[NSUIHeartBeat_heartBeatThread:]
> > 3 forThreadForFunction
> > 4 _pthread_body
> >
> > Thread-4
> > 0 mach_msg_overwrite_trap
> > 1 mach_msg
> > 2 _pthread_become_available
> > 3 pthread_exit
> > 4 _pthread_body
> >
> > This doesn't tell me more than that the app crashes due to a recursion
> > that never ends - which I already knew.

>
> Actually, it tells you a lot of valiable information.

For those who know how to read it. B)

> I think you
> snipped a little too much,

Nope. Thread-3 & -4 are complete. It's all I got in the debugger.

> but that's ok. (And apparently you were
> typing this out by hand?

It's that obvious? ;)

> Try typing "bt" in the Console tab, then
> copy/paste the output if you need to send another call stack.)

That was helpful. Thanks. B)

> Basically, from here you can see exactly where the recursion is
> occuring, although it doesn't necessary look like anything new. It looks
> like, as we thought, that A encodes B encodes A repetitively; but this
> is somehow happening through encodeConditionalObject:. If you click on a
> function in the call stack and the function is one of yours, the
> debugger will jump to the source for that function and let you examine
> the state of all the variables, and see exactly where you were at the
> time. That might help.

As far as I can tell, it goes straight from RootClass to the first
instance of ClassA and then dives right into the recursion. But then
I'm not sure what I'm looking for.

However, if I go to the [RooTClass encodeWithCoder:] and then check
the _objc_super variables, I get a lot of "Cannot access memory at
address [hexadecimal]" errors in the console. The ClassA & B instances
have a "protocol_list" that gives the same message - although the list
in the first instance of A is only one deep, the one in the first
instance of B had "folders"(?) nested deeper than the width of the
window would let me go.

(delayed consequence)

> Anyway, this is a big digression, since your problem is fortunately a
> lot more straightforward than that. :)

I wish it felt straightforward to me. B)

--
C Lund, [url]www.notam02.no/~clund[/url]