[GAP] Effect of using malloc for temporary allocations in GAP
caj21 at st-andrews.ac.uk
Mon Jan 2 14:44:39 GMT 2017
My belief is the follows (we can find out).
The main problem is contiguous address space (both the master pointers, and the allocated memory, must be in one giant block for gasman), and the fact we can't move master pointers.
On 64-bit we fairly early reserve a massive piece of address space for gasman (not actually Malloced, just reserved) after which malloc can do whatever it likes. This can cause problems if users care about overcommit, but let's ignore such users.
On 32-bit OSes, we grab some memory then try to extend later. In principle any malloc can cause any later attempt to extend memory space to fail. I suggest we move to grabbing a bigger piece of memory space early, then we can also malloc freely. There is a choice to be made on the trade-off on how address space to give to gasman, and how much to give to malloc, but I think nowadays we can reasonably say for any sizable calculation users should run 64-bit GAP.
We could, for example, say on 32bit OSes we will set gap's max address space to 512MB, and grab that much address space for gasman early (we would have to do some experiments on different OSes to see what is reasonable).
Get Outlook for Android<https://aka.ms/ghei36>
On Mon, Jan 2, 2017 at 2:08 PM +0000, "Max Horn" <Max.Horn at math.uni-giessen.de<mailto:Max.Horn at math.uni-giessen.de>> wrote:
> On 02 Jan 2017, at 15:01, Dmitrii Pasechnik wrote:
> I would like to point out that GMP allows one to specify her/his own
> memory allocator: https://gmplib.org/manual/Custom-Allocation.html
> (I'm surprised that GAP does not use it, or does it?)
GAP cannot use these hooks, because the GAP GC is a moving GC. That means that
it could happen that, after making an allocating, the actual location of
an object (say, the inputs to the GMP function currently running) move. Since
GMP is not aware of this, and there is no way to teach it about this (other than
modifying its code, of course), a crash and/or corrupt data would result.
Of course we could "fix" this by changing our GC: Either by replacing it by
a different GC (the HPC-GAP uses Boehm GC, but for various reasons this
causes a noticeable speed decrease), or by enhancing the existing GC; but doing
that is a major undertaking.
Prof. Dr. Max Horn
Tel: (+49) 641 99-32044
Fax: (+49) 641 99-32049
E-Mail: max.horn at math.uni-giessen.de
Gap mailing list
Gap at gap-system.org
More information about the Gap