I have some concerns about the driver's memory management when using MakeBufferResidentNV. My understanding is that this function locks a buffer at its present VRAM location in order to provide a constant GPU address. If that is the case, this would seem to exacerbate the memory fragmentation of VRAM. If some randomly sized buffers are locked at some random positions in memory, the driver won't be able to defrag the memory and later buffer allocations will eventually fail. How will the GL cope with this?
Regarding structures made of pointers, maybe it would be a good idea to add refcounts to resident GPU addresses. As it stands now, the application has to do this on its own in order to use a resident GPU address in multiple structures. It is really only safe to delete a resident buffer shared across multiple structures via its GPU address when its usage count drops to 0. Otherwise, it generates dangling pointers to deleted memory. Essentially, it seems to me that using GPU addresses shared across structures is an easy way to introduce hard to debug crashes.