[GAP] Effect of using malloc for temporary allocations in GAP
Max.Horn at math.uni-giessen.de
Mon Jan 2 17:05:21 GMT 2017
> On 02 Jan 2017, at 15:44, Christopher Jefferson <caj21 at st-andrews.ac.uk> wrote:
> My belief is the follows (we can find out).
> The main problem is contiguous address space (both the master pointers, and the allocated memory, must be in one giant block for gasman), and the fact we can't move master pointers.
BTW, I just discovered the file "dev/sacks.tex" in our repository, written
by Steve, which discusses extending GASMAN to cover with non-contiguous
> On 64-bit we fairly early reserve a massive piece of address space for gasman (not actually Malloced, just reserved) after which malloc can do whatever it likes. This can cause problems if users care about overcommit, but let's ignore such users.
I believe this is what the default value for SyAllocPool and SyStorMax are meant to achieve (in InitSystem). If I read this correct, on 64bit systems we reserve 4GB and on 32bit systems (other than cygwin) 1.5 GB. On Cygwin32, we invoke malloc and free a couple times, presumably to achieve a somewhat similar effect (though curiously, the comment in lines 2068ff only talks about GNU clib <sic>).
> On 32-bit OSes, we grab some memory then try to extend later. In principle any malloc can cause any later attempt to extend memory space to fail. I suggest we move to grabbing a bigger piece of memory space early, then we can also malloc freely. There is a choice to be made on the trade-off on how address space to give to gasman, and how much to give to malloc, but I think nowadays we can reasonably say for any sizable calculation users should run 64-bit GAP.
> We could, for example, say on 32bit OSes we will set gap's max address space to 512MB, and grab that much address space for gasman early (we would have to do some experiments on different OSes to see what is reasonable).
My interpretation of the code is that we even grab 1.5 GB... but I only glanced through it, did not try to grok it in detail (it's fairly convoluted).
The fact that readline and gcd computations already potentially perform (temporary) mallocs make me hopeful that this all indeed works, and it is worthwhile to investigate using some of the "high level" GMP functionality.
Prof. Dr. Max Horn
Tel: (+49) 641 99-32044
Fax: (+49) 641 99-32049
E-Mail: max.horn at math.uni-giessen.de
More information about the Gap