[GAP] Effect of using malloc for temporary allocations in GAP

Max Horn Max.Horn at math.uni-giessen.de
Mon Jan 2 17:24:15 GMT 2017


Hi Dima,

> On 02 Jan 2017, at 17:18, Dmitrii Pasechnik <Dmitrii.Pasechnik at cs.ox.ac.uk> wrote:
> 
> Dear all,
> On Mon, Jan 02, 2017 at 03:07:34PM +0100, Max Horn wrote:
>>> On 02 Jan 2017, at 15:01, Dmitrii Pasechnik <Dmitrii.Pasechnik at cs.ox.ac.uk> wrote:
>>> I would like to point out that GMP allows one to specify her/his own
>>> memory allocator: https://gmplib.org/manual/Custom-Allocation.html
>>> (I'm surprised that GAP does not use it, or does it?)
>> 
>> GAP cannot use these hooks, because the GAP GC is a moving GC.
> 
> Well, these hooks allow one to bundle on a custom malloc with address space
> protected from GC.

The issue is not protecting the address space from our GC. It is more about
the reverse, i.e. "protecting" our GC from an allocation in a place that
prevents the GASMAN heap from growing.


> There is probably quite a big choice of custom
> mallocs to pick, and very small, or none, changes to the GC needed...

"Probably"? That's a bold statement to make -- I am not aware of any (unless
you count replacing GASMAN by Boehm GC, which technically is a "custom
malloc"). And I have a hard time even imagining a way how a 3rd party
allocator, which is not aware of GASMAN, would pull that off without major
changes to GASMAN and/or that allocator.  (unless that custom allocator did
something serious heavy lifting including stack and register walks etc, like
Boehm GC).

But of course that may be entirely due to my lack of imagination!  So, if
you happen to know how to do this, a patch would be highly welcome!  This is
a problem we've been facing for many years, and which prevents us from
making better use of 3rd party C/C++ libraries. So, there is a high desire
to overcome it, and if you have a genius way to overcome it, all praise to
you!

Of course this should ideally be done without compromising the performance
of existing code, of course. Which is why the only implemented "solution" I
am aware of, i.e. using Boehm GC, is currently not that attractive. Though
of course the situation is more complex, esp. if one also factors in HPC-GAP
support.

> 
>> That means that
>> it could happen that, after making an allocating, the actual location of
>> an object (say, the inputs to the GMP function currently running) move. Since
>> GMP is not aware of this, and there is no way to teach it about this (other than
>> modifying its code, of course), a crash and/or corrupt data would result.
> 
> Are you talking about multithreading?

No.

> Unless GMP uses absolute address arithmetic, and assuming GC knows its
> pointers, GMP can be paused, the pointers rewritten by GC as needed,
> and then resumed.

I am not sure what you mean with "absolute address arithmetic" -- but if you
mean that it uses raw pointers just like virtually every other C library
does, then: Yes, of course it does that. In particular, our GC does *not*
know where it stores these raw pointers, so it cannot rewrite them.

Of course Boehm GC deals with that, but this requires *heavy* lifting,
including intricate knowledge about the stack layout of the target
architecture. At this point, the only solution I see is a major overhaul of
GASMAN or switching to a new GC (either written from scratch for our needs,
or imported from another project with similar needs; but that, too, has some
major pitfalls). Either is a major change and not something undertaken
lightly. I know that Reimer worked in this direction on and off, but it's
not part of his current job, and in any case not something you just do on
the side...

Note that I think improving GASMAN and/or switching to a new GC, both while
not seriously affecting performance, and not impending multi-thread support,
is possible, and indeed desirable. Heck, I'd even love to work on it, except
that doing it would destroy my career. So I won't... But if I ever get
tenure, and nobody solved this by then, feel free to remind me about it :-)


Bye,
Max
-- 
Prof. Dr. Max Horn
AG Algebra
Mathematisches Institut
Justus-Liebig-Universität Gießen
Arndtstraße 2
D-35392 Gießen
Tel: (+49) 641 99-32044
Fax: (+49) 641 99-32049
E-Mail: max.horn at math.uni-giessen.de




More information about the Gap mailing list