As of yet, my renderer has not handled streaming of vertex and index data. This is ok for small and mid-sized scenes where nothing too special happens. In its current state, a mesh will request to be uploaded to a buffer and stay there until program termination. The maximum buffer size is of course flexible and if not enough memory is left for a particular mesh, the buffer manager will allocate a new vertex and index buffer and put it there. To ensure minimal state changes, especially for VAOs, the sorting priorities are chosen accordingly, so stuff is sorted by VAO, then by shader and then, if possible by textures, unless the last step will incur more sorting cost than it's worth. In general, meshes are rendered with base vertices and assembled to be multi-drawable. However, either single instances with DrawElementsBaseVertex or multiple instances are trivial either.
Now, streaming stuff is a whole different story. I was thinking about keeping static (or very infrequently evicted) data and very dynamic in a separate buffers which would mirror the GL's buffer usage scenarios (although performance-wise this does not necessarily advantageous). Leave the static buffers alone most of the time and only fiddle with the dynamic ones. However, this is not the real problem. The real problem is fragmentation over time since you can't expect that meshes are evicted in reverse order. This wastes a lot of memory in small chunks if you can't fill the gaps anymore. First of all, I'd probably want to handle free memory sort of like Alexandrescu using a list of chunks, which would include an offset and a chunk size, with the first chunk holding 0 and the buffer size. Then, there'd be multiple possibilities of handling fragmentation, so if a certain threshold of fragmentation is surpassed I could:
a) reorganize the buffer, i.e copy data around and make it contiguous again
b) create a new buffer and upload formerly evicted meshes to that new buffer while slowly draining the old one until it is below the threshold or empty
c) track the number of successful uploads over time and then see if fragmentation is too high (or if the buffer is simply full), and do either a) or b)
Granted, I may be overreaching here a little bit. Please keep in mind, this is not meant for a simple demo or something, but as a streaming system for a heavy-duty renderer. What's your experience with this? Any suggestions?