[GAP] Effect of using malloc for temporary allocations in GAP
Dmitrii.Pasechnik at cs.ox.ac.uk
Mon Jan 2 22:09:04 GMT 2017
On Mon, Jan 02, 2017 at 08:59:46PM +0100, Max Horn wrote:
> > On 02 Jan 2017, at 19:31, Dmitrii Pasechnik <Dmitrii.Pasechnik at cs.ox.ac.uk> wrote:
> > On Mon, Jan 02, 2017 at 06:24:15PM +0100, Max Horn wrote:
> >>> On 02 Jan 2017, at 17:18, Dmitrii Pasechnik <Dmitrii.Pasechnik at cs.ox.ac.uk> wrote:
> >>> On Mon, Jan 02, 2017 at 03:07:34PM +0100, Max Horn wrote:
> >>>>> On 02 Jan 2017, at 15:01, Dmitrii Pasechnik <Dmitrii.Pasechnik at cs.ox.ac.uk> wrote:
> >>>>> I would like to point out that GMP allows one to specify her/his own
> >>>>> memory allocator: https://gmplib.org/manual/Custom-Allocation.html
> >>>>> (I'm surprised that GAP does not use it, or does it?)
> >>>> GAP cannot use these hooks, because the GAP GC is a moving GC.
> >>> Well, these hooks allow one to bundle on a custom malloc with address space
> >>> protected from GC.
> >> The issue is not protecting the address space from our GC. It is more about
> >> the reverse, i.e. "protecting" our GC from an allocation in a place that
> >> prevents the GASMAN heap from growing.
> > Why does GASMAN use the heap (managed by (s)brk/malloc) as opposed to a separate virtual memory
> > region which one can obtain using mmap?
> > Or am I missing something?
> GASMAN already uses virtual memory APIs were available (I used "heap" in the
> general sense, i.e. "dynamically allocated, not on the stack"). But this
> does not solve the problem, it just shifts it -- the problem is that GASMAN
> needs a contiguous memory region right now, and a single "foreign" page
> allocation can prevent GASMAN from extending its memory.
It seems that also other processes on the system may cause such a
situation, where the continuity of the free address space is broken, right?
Looks like one is pretty much at the mercy of the OS here.
> On 64bit systems,
> pre-reserving space works pretty well, as the virtual address space is much
> larger than the real memory (though even then you could be "unlucky", it's
> just not very probably). But on 32bit, the fact that the address space is
> limited to 4 GB makes this trick much less useful. See also the mails by me
> and Chris in this thread.
> In any case, this all is completely besides the point of my original question.
IMHO it's still quite relevant: we seem to have established that the
system mallocs happen in another VM region, and so they interfere
with GASMAN no more than other, outside of our control, processes.
Fixing GASMAN so that it can use non-continuous address space seems
to the the only real way out.
> Prof. Dr. Max Horn
> AG Algebra
> Mathematisches Institut
> Justus-Liebig-Universität Gießen
> Arndtstraße 2
> D-35392 Gießen
> Tel: (+49) 641 99-32044
> Fax: (+49) 641 99-32049
> E-Mail: max.horn at math.uni-giessen.de
More information about the Gap