async texture loading

I have main thread that renders some graphics and another thread that loads textures using separate GL context that is shared with main one. This works perfectly on Linux but is slow on Windows. Does anybody knows what’s the problem and how to load textures not interrupting main thread??

I don’t know why it’s slow on Windows, but you don’t need a separate context just to load textures asynchronously in the first place! Why not load the image into a local buffer, using your loader thread, then signal the main thread, when data is ready, and then load data using glTexImage from your main thread? A more advanced solution would be to load straight into a PBO, but that’s optional…

err, i’ve sent this message twice by accident… is it possible to remove this second one somehow?

I don’t know why it’s slow on Windows, but you don’t need a separate context just to load textures asynchronously in the first place! Why not load the image into a local buffer, using your loader thread, then signal the main thread, when data is ready, and then load data using glTexImage from your main thread? A more advanced solution would be to load straight into a PBO, but that’s optional…

the problem is that glTexImage2D itself takes much time and unacceptable in main thread because image needs to be updated smoothly (camera move)

Ive found the same problem. glTexImage takes a lot of time. glTexSubImage also makes no improvement even though i though it would.
Ive seen programs like the free WorldWind from NASA thats seems to do this seamlessly…

If speed is an issue, then load data into PBO first. glTexImage from a PBO is lightning fast. If it’s still not fast enough, then just upload one texture (or even just a subimage) each frame.

As for WorldWind, I believe it uses DXT1 compressed textures. A 2048x2048 RGB texture is just 2MB when compressed with DXT1! Uploading 2MB is a snap! How much data you are trying to upload every frame? I’d also guess that WorldWind doesn’t switch all the textures at once, rather it loads them up as required, one at a time. Of course, these are only guesses from my part. The WorldWind project is open source, so you could take a look at how exactly they did it…

I know about WorldWind but it’s DirectX based and not OpenGL, so it won’t help me much. Btw, it also updates models and loads textures in separate thread. And I’m wondering why using separate thread doesn’t work for me.
Very good example is Google Earth - it’s like WorldWind but OpenGL based (and not opensourc:( ), camera move is very smooth there and isn’t interrupted at all by texture loading so I’m very interesting in how this is done.

finally I found the solution!! :slight_smile:

i tunred off S3TC and FXT1 texture compression in driver options in Windows and now my app works as fast as on Linux:)

so next question (I’m new to OpenGL programming) - how to control these settings from my app?

The specs :
http://oss.sgi.com/projects/ogl-sample/registry/EXT/texture_compression_s3tc.txt
http://oss.sgi.com/projects/ogl-sample/registry/ARB/texture_compression.txt
Sample code :
http://www.robthebloke.org/opengl_programming.html (search for ‘compression’ in the page)

I know about WorldWind but it’s DirectX based and not OpenGL, so it won’t help me much.
It doesn’t really matter if it’s OpenGL or not, what I said about loading still holds.

Btw, it also updates models and loads textures in separate thread.
I’ve never said otherways. What I said is that you do not need a separate context to do this. :slight_smile:
And don’t forget about the powerful PBOs!!

but how can two threads use one context at the same time?

in any case now it works quite well for me (my textures are small and I don’t need PBOs and so on, it was slow because of texture compression done by driver).

but how can two threads use one context at the same time?
They can’t. You only upload textures from the one thread that has the context. But the other thread can still do all the hard work without needing a context. I’ve explained it before, please read more carefully. :wink:
Anyway, if it works for you now it’s fine, I’m just saying that IMHO it’s not worth the trouble to have multiple contexts for this. You can do simpler and better.

i havent seen Google Earth or WorldWind but i can imagine what theyre like.
loading a 1024x1024 dxt3/5 texture takes a LOT less than 0.01 seconds thus u can happily maintain 100fps on a reasonable system by just loading one texture a frame (ive done this myself) with absolutly no pauses when new textures are loaded in.
no threads are needed

but I can see pauses… and they aren’t there when no textures need to be loaded…

ok, thank you, I’ll play with texture formats/flags…

but I can see pauses… and they aren’t there when no textures need to be loaded…
Can you tell us how many, and what size/format textures you are trying to upload every frame?

BTW: PBO! PBO! PBO! :smiley:

textures are small - 512x512, I’m loading them from .jpg files (in separate thread) and then uploading as GL_RGB, now I can see very small difference (if any) between uploading in main thread, one per frame, or in separate thread (and context). of cource maybe there’s something wrong in other parts of my code, but I surely can see difference between situations when new textures need to be uploaded and when not.

any tutorial/doc on using PBO?

Could you try uploading in main thread, one per frame, but do the jpeg decompression in another thread? If it’s still slow, I suspect there is something else.

As for PBOs, you won’t find too many tutorials, but it’s exactly the same as a VBO, for which you’ll find tons of materials online. The only difference is that with PBO, you can bind an arbitrary buffer object to GL_PIXEL_UNPACK_BUFFER target, and after this, every glTexImage call will source the data from the bound buffer object. And the contents of the buffer object can be loaded in a separate thread, and it’s usually located in VRAM or AGP memory, from where the actual upload with glTexImage will be lightning fast. Hope this helps.

If you use glTexImage and S3TC the driver compresses you textures --> slow

If you use glCompressedTexImage with precompressed S3TC format and proper parameters for the call, the driver just copies the texture to the GPU --> fast

(I just played a bit with Google Earth : they seem to use only small textures)