persistent malloc

[Luiz Henrique de Figueiredo]

I'm looking for a malloc replacement that uses a persistent mmap'd file, so
that malloced data structures persist from one run to the other. I've tried
mmalloc, which comes with gdb, and it works mostly but I get some core dumps,
which don't happen when I use malloc from libc.

Does anyone have experience with mmalloc and persistency?
Is there a simpler or better alternative?

Thanks for any info.

[David Schwartz]

"Luiz Henrique de Figueiredo" <lhf@csgpwr1.uwaterloo.ca> wrote in message
news:bg763e$pum$1@tabloid.uwaterloo.ca...

> I'm looking for a malloc replacement that uses a persistent mmap'd file,

so

> that malloced data structures persist from one run to the other.

I don't follow. Can you be more precise about what you're trying to do.

> Does anyone have experience with mmalloc and persistency?
> Is there a simpler or better alternative?

Decide on what data you want to save across runs, decide on a format for
that data, and write it to a file. If you're using C++, store the data you
need in serializable classes.

DS

[Luiz Henrique de Figueiredo]

In article <bg78m7$26f$1@nntp.webmaster.com>,
David Schwartz <davids@webmaster.com> wrote:

>
> I don't follow. Can you be more precise about what you're trying to do.

I want all my malloc'd data to be saved into a file, pointers and all, and I
want to be able to load the file in each run and get the same data structures,
pointers and all, as in the previous run. I don't want serialization. I want
to lazy route. :-)

The mmalloc library that comes with gdb provides a malloc replacement that
uses a mmap'd file as its pool. The file persists across runs and does what I
want, expect that I get some seg faults that I don't get when using the
default malloc from the C library.

For infor on mmalloc, see [url]http://sources.redhat.com/gdb/onlinedocs/mmalloc.html[/url]

So, does anyone have experience using mmalloc for this?

Thanks.

[David Schwartz]

"Luiz Henrique de Figueiredo" <lhf@csgpwr1.uwaterloo.ca> wrote in message
news:bg79je$rtb$1@tabloid.uwaterloo.ca...

> In article <bg78m7$26f$1@nntp.webmaster.com>,
> David Schwartz <davids@webmaster.com> wrote:

> > I don't follow. Can you be more precise about what you're trying to

do.

> I want all my malloc'd data to be saved into a file, pointers and all, and

I

> want to be able to load the file in each run and get the same data

structures,

> pointers and all, as in the previous run. I don't want serialization. I

want

> to lazy route. :-)

That's not really possible in principle. What if the address one of
those pointers points to is already in use at the time your process attempts
to attach to the file?

> The mmalloc library that comes with gdb provides a malloc replacement that
> uses a mmap'd file as its pool. The file persists across runs and does

what I

> want, expect that I get some seg faults that I don't get when using the
> default malloc from the C library.

I'm not surprised you get segfaults. The pointers probably don't point
to the right place in the new process.

You can make this work, but all of your pointers must be relative to the
base of the file. There are other requirements as well.

DS

[Gianni Mariani]

> That's not really possible in principle. What if the address one of
> those pointers points to is already in use at the time your process attempts
> to attach to the file?
>
>

>>The mmalloc library that comes with gdb provides a malloc replacement that
>>uses a mmap'd file as its pool. The file persists across runs and does

>
> what I
>

>>want, expect that I get some seg faults that I don't get when using the
>>default malloc from the C library.

>
>
> I'm not surprised you get segfaults. The pointers probably don't point
> to the right place in the new process.
>
> You can make this work, but all of your pointers must be relative to the
> base of the file. There are other requirements as well.
>

I wrote a malloc (also called MMmalloc) that has a mode were all the
pointers in the arena are offsets and can be mmapped anywhere in the
process space.

I've placed it here:

[url]http://www.mariani.ws/~gianni/MMmalloc.tar.gz[/url]

It's way old but it's easy enough to extract the interesting bits (being
how to create a relocatable arena).

G

[James Antill]

On Wed, 30 Jul 2003 02:13:02 +0000, Luiz Henrique de Figueiredo wrote:

> In article <bg78m7$26f$1@nntp.webmaster.com>,
> David Schwartz <davids@webmaster.com> wrote:

>>
>> I don't follow. Can you be more precise about what you're trying to do.

>
> I want all my malloc'd data to be saved into a file, pointers and all, and I
> want to be able to load the file in each run and get the same data structures,
> pointers and all, as in the previous run. I don't want serialization. I want
> to lazy route. :-)

The problem here is global variables, say you have...

static struct store *beg = NULL;

static struct store *store_add(void *data);
static void store_del(struct store *store);
static struct store *store_srch(void *data);

Now everything inside the store_* API should be fine, assuming you can
easily restore the memory via. MAP_FIXED etc. However the "beg" variable
isn't known the the malloc() calls, so you need to register all
static/global variables so you can restore them (I've seen this done and
it's _far_ from the lazy route) ... As a corner case of this you could
probably use a single global and have everything else use a context that
is passed to them, but this is only feasable if almost everything already
takes a context.

Also note that in both cases you can never call anything like
gethostbyname() etc. unless you're malloc replacement has different
function names than malloc() and free().

--
James Antill -- [email]james@and.org[/email]
Need an efficent and powerful string library for C?
[url]http://www.and.org/vstr/[/url]