Creating a scalable universe

Working on creating a universe like model, where there could be millions of objects (data points). I currently place all my vectors for the objects, colors, etc into a VBO. But how large can this scale… I expect that you really don’t want to draw more than a 100,000 objects in the scene and fade/clip out the rest. But at some point, I have to be able to move through that universe and access the other million objects. I’ll need to search those data points, and fly to them. Has anyone had any experinece with how you would go about refreshing the VBO as you move through space like that. Perhaps using a Graph database like Neo4j or OrientDB? I’m not sure that would contain the vectors… how would you know which data points to load/draw? Would you pull the entire vector data set into memory or the VBO, then use elements? Thanks for any direction you can provide.

A basic system is to divide your universe in cubes, X lightyears wide.
At any moment, draw the 8 cubes closest to the camera. Each Cube is a VBO. That way you keep the number of both updates and draw calls low, for static stuff.
More subtle is to take in account the data point luminosity, and do “sort of” geometry mipmaps. Larger cubes contain only brighter points, etc.

In a 3d world, would that be 23 cubes - cube for every direction? Also, how would you go about indexing all your vertex points so that you know what to load into the surrounding cubes. Would you precalculate and assign each vertex to a cube and then just keep track of your cube position, reloading the edge cubes as you move?

Thanks for your help!

I think that would actually be 27 cubes(I call them chunks) for a fully 3D world including the chunk at the center.

I’m doing the same thing on the terrain level, but with a 5x5 chunks instead of 3x3 but same concept. I always start at position (0,0) and simply translate the size of each chunk along the x and z when creating my display lists(migrating to VBO currently.) Each chunk has it’s coordinates relative to itself only (I believe this is called “Model/Object Space”). So in the case of my grid style terrain, all the coordinates are exactly the same for each chunk except the heights.

By keeping track of your current position and knowing the size of each chunk, you can do some simple math to calculate which chunk you are in and your relative coordinates for that chunk. That information can be used to dynamically load new surrounding chunks.

I think thats the information you were trying for, hope that it’s all relevant and helpful to what you are trying to do.

Ok, so lets say I’ve created this graph universe and the points (Nodes) are broken up into cubes and I can navigate through this space. What are the thoughts on link lines? Currently, all my link lines exist as elements pointing to the vertex VBO. So as I travel through space in this new model, some cubes will be loaded and some not. So how can I draw link lines that may extend to areas not loaded. Not sure how to approach this…

I was thinking that if the links are outside of the visible cubes, I could then switch to designating the outside cubes as a vertex points. Since my lines fade with distance, you would only see the destination as you fly to it. Does this sound like a reasoned approach, or do you have any suggestions on how I should attack this.

you could just split the lines at cube boundaries so that a line crossing multiple cubes actually consists of a number of line segments and there is no geometry that crosses cube boundaries.

That is only really feasible if the number of objects that normally would cross a boundary (and thus require splitting) is small, otherwise it likely becomes a pain to manage.