Part of the Khronos Group
OpenGL.org

The Industry's Foundation for High Performance Graphics

from games to virtual reality, mobile phones to supercomputers

Results 1 to 7 of 7

Thread: glMapBuffer(...) returns null pointer if another application occupied first 32G ram

  1. #1
    Junior Member Newbie
    Join Date
    Jun 2017
    Posts
    9

    glMapBuffer(...) returns null pointer if another application occupied first 32G ram

    We have two applications A and B.
    A is a CUDA computing app, allocate huge pinned memory.
    B is an OpenGL based app.
    The whole system host memory is 96G. Using GTX 1070.
    When app A allocates more than 32G pinned memory, function glMapBuffer(...) just returns null pointer as there is no virtual memory. But definitely there is.
    If app A allocates less than 32G pinned memory, then glMapBuffer(...) works properly.

    My question: is there a limitation for OpenGL's memory management? For example, it needs some buffer memory like page-locked memory to do data transfer and this part of memory must be located in the first 32G memory.

  2. #2
    Senior Member OpenGL Pro
    Join Date
    Jan 2007
    Posts
    1,714
    OpenGL itself doesn't specify this kind of behaviour. This is going to be implementation-dependent.

    Note the documentation for glMapBuffer: https://www.khronos.org/registry/Ope...lMapBuffer.xml

    If the GL is unable to map the buffer object's data store, glMapBuffer generates an error and returns NULL. This may occur for system-specific reasons, such as low virtual memory availability.

  3. #3
    Junior Member Newbie
    Join Date
    Jun 2017
    Posts
    9
    Quote Originally Posted by mhagain View Post
    OpenGL itself doesn't specify this kind of behaviour. This is going to be implementation-dependent.

    Note the documentation for glMapBuffer: https://www.khronos.org/registry/Ope...lMapBuffer.xml

    Hi mhagain, thank you for the reply.
    Actually I noticed this line and was wondering why there is not enough virtual memory. Our virtual memory size is 96G.
    That's why I guess OpenGL's driver under win7 cannot recognize the memory larger than 32G.
    All the experiments proved that the pinned memory in app A must be lower than 32G to allow app B works properly.

  4. #4
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    4,152
    Quote Originally Posted by lzheng View Post
    The whole system host memory is 96G. Using GTX 1070.
    When app A allocates more than 32G pinned memory, function glMapBuffer(...) just returns null pointer as there is no virtual memory. But definitely there is. If app A allocates less than 32G pinned memory, then glMapBuffer(...) works properly.
    Ok first, glMapBuffer() returning NULL doesn't necessarily mean there is no remaining virtual memory. But (on NVidia drivers), it may indicate that there is insufficient "pinned memory" left to satisfy the request.

    AFAIK, the NVidia driver locates GL buffer objects either in GPU (on-device) memory or in CPU pinned memory (you can easily see this if you install a message callback using glDebugMessageCallback() -- see this wiki link), and under some circumstances, they can migrate from one memory space to the other. Mapping a buffer on NVidia GL drivers automatically demotes the buffer object to pinned memory by default (IIRC), if it wasn't already there.

    Pinned memory is a special type of virtual memory (as you probably already know) which is not paged. Other names for "pinned memory" include: page-locked memory (on Linux) or non-paged memory or non-paged pool (on Windows). The opposite type of virtual memory is of course Paged memory (i.e. memory that can be paged out of physical RAM).

    My question: is there a limitation for OpenGL's memory management? For example, it needs some buffer memory like page-locked memory to do data transfer and this part of memory must be located in the first 32G memory.
    For OpenGL in general, no (OpenGL is a spec defining an API, not a driver implementation). However, a specific GL driver could potentially have a limitation like this.

    ...that said, on a 64-bit CPU (which your numbers suggest you're running on), I've not read or heard of a general limitation pn the amount of pinned memory being that low. In fact, on the net you can find folks that have much more pinned memory than that reserved.

    How much non-paged memory is your Window 7 boxed configured for, and what does it show for the limit? (see Task Manager -> Performance -> Kernel Memory : Nonpaged, and/or Process Explorer -> System Information -> Memory). This is typically going to be a small subset of your total virtual memory.

    Also, what's your "NonPagedPoolSize" set to (not sure if setting this on Win7 works or not)?:

    * NonPagedPoolSize

    Check out Process Explorer or poolmon for tools that should let you monitor your non-paged pool usage dynamically.

    According to this Microsoft blog post and reference page, the max non-paged pool memory on Win7 64-bit is min( 75% * Amount of RAM, 128 GB):

    * Pushing the Limits of Windows: Paged and Nonpaged Pool
    * Memory Limits for Windows and Windows Server Releases

    so with the right config, you should be able to fix your problem (**), collectively allocating at most 96GB of RAM for pinned memory with your current installed RAM.

    (**) ...that is, unless there's some limitation in CUDA or the NVidia drivers that you hit up against.
    Last edited by Dark Photon; 06-17-2017 at 10:00 AM.

  5. #5
    Senior Member OpenGL Pro
    Join Date
    Jan 2007
    Posts
    1,714
    the main point I guess is that this is going to be implementation-dependent.

    A GTX 1070 is a consumer-class card, so (and I'm really entering speculation land here) I would suggest that it is neither designed for nor tested with a data set of this size. This problem most likely would not occur if using a Quadro.

  6. #6
    Junior Member Newbie
    Join Date
    Jun 2017
    Posts
    9
    Quote Originally Posted by Dark Photon View Post
    Ok first, glMapBuffer() returning NULL doesn't necessarily mean there is no remaining virtual memory. But (on NVidia drivers), it may indicate that there is insufficient "pinned memory" left to satisfy the request.

    AFAIK, the NVidia driver locates GL buffer objects either in GPU (on-device) memory or in CPU pinned memory (you can easily see this if you install a message callback using glDebugMessageCallback() -- see this wiki link), and under some circumstances, they can migrate from one memory space to the other. Mapping a buffer on NVidia GL drivers automatically demotes the buffer object to pinned memory by default (IIRC), if it wasn't already there.

    Pinned memory is a special type of virtual memory (as you probably already know) which is not paged. Other names for "pinned memory" include: page-locked memory (on Linux) or non-paged memory or non-paged pool (on Windows). The opposite type of virtual memory is of course Paged memory (i.e. memory that can be paged out of physical RAM).



    For OpenGL in general, no (OpenGL is a spec defining an API, not a driver implementation). However, a specific GL driver could potentially have a limitation like this.

    ...that said, on a 64-bit CPU (which your numbers suggest you're running on), I've not read or heard of a general limitation pn the amount of pinned memory being that low. In fact, on the net you can find folks that have much more pinned memory than that reserved.

    How much non-paged memory is your Window 7 boxed configured for, and what does it show for the limit? (see Task Manager -> Performance -> Kernel Memory : Nonpaged, and/or Process Explorer -> System Information -> Memory). This is typically going to be a small subset of your total virtual memory.

    Also, what's your "NonPagedPoolSize" set to (not sure if setting this on Win7 works or not)?:

    * NonPagedPoolSize

    Check out Process Explorer or poolmon for tools that should let you monitor your non-paged pool usage dynamically.

    According to this Microsoft blog post and reference page, the max non-paged pool memory on Win7 64-bit is min( 75% * Amount of RAM, 128 GB):

    * Pushing the Limits of Windows: Paged and Nonpaged Pool
    * Memory Limits for Windows and Windows Server Releases

    so with the right config, you should be able to fix your problem (**), collectively allocating at most 96GB of RAM for pinned memory with your current installed RAM.

    (**) ...that is, unless there's some limitation in CUDA or the NVidia drivers that you hit up against.

    Thank you so much for the reply.
    I was disturbed by other thing for a while. Sorry for the delayed response. I really appreciated the detailed answer and want to do some testing before I asked you more questions which also takes some time.

    Following the answer, I read the two links you provided. It seems that default nonpaged pool size is 75% of RAM or 128GB. The paged pool size in win 7 is min(system commit limit, 384GB).

    In Registery Editor, the NonPagedPoolSize is a DWORD, limited to 32bit, I cannot make it larger than 4GB.
    According to Process Explorer, this is how our win 7 with 32GB RAM or 96GB RAM look like:
    Click image for larger version. 

Name:	KernelMemory.png 
Views:	23 
Size:	3.0 KB 
ID:	2413
    So the operating system's default Nonpaged Limit is 16,777,216
    This confused me.
    Last edited by lzheng; 07-14-2017 at 02:51 PM.

  7. #7
    Junior Member Newbie
    Join Date
    Jun 2017
    Posts
    9
    So now it seems to be Nvidia driver's problem.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •