Do not access DevIL's functions from multiple threads

Although this is not OpenGL related post, I think it is worth mentioning here and warn community, especially beginners, that DevIL image manipulation library has a very bad design, and does not allow access from multiple threads.

The reason is simple, it attempts to mimic OpenGL and the way it handles buffers, but there is a crucial difference; OpenGL allows just one thread to access a single context. DevIL does not have context, and the state is global. Setting current image as a global state is meaningless in moder applications. The only way (I can see) to make DevIL threadsafe is to add all image manipulation functions in a singleton object. For me, this is also a bad design, but at least an application will not crash.

My attempt to transfer the code to use DevIL was a total failure. I’ll back to GDI+ definitely.

What do you suggest as a cross platform image manipulation library?

Was DevIL an image manipulation library? What kind of manipulation were you trying to do?

The only way (I can see) to make DevIL threadsafe is to add all image manipulation functions in a singleton object.

I don’t see how that makes it thread-safe. Wrapping DevIL calls in a singleton does nothing to stop you from calling into that singleton from multiple threads. A bigger question might be why you need multithreaded image loading.

@Alfonse: Usually while you load a large scene a lot of time is spent on disk reading, and you have OPEN… READ… DO SOMETHING… READ… DO SEMETHING ELSE… CLOSE OPEN ANOTHER FILE… REPEAT FROM 2, and this is SLOOOOW. Once you have read the disk you can decompress image and generate mipmap with multiple thread. One thread for reading raw data from the disk and minimize disk seek store them on a buffer and spawn a work for image manipulation.

Last time I made something similar I have used openIL to read images from a working thread. Once the geometry was loaded I started works on another to load the pictures. I made only ONE thread to load pictures, once finished a callback create an openGL texture (on another shared context) and inform the scene renderer that the texture is ready to use. When I tried to spawn multiple working thread I got a lot of multithread issue and when I tried to remove them with lock basically all works worked sequentially, so I returned with a single working thread doing stuff.

About the thread safe design. When you create a library you have to decide to make it thread safe or to make the user to make it used thread safe. The latter is used when you create that must be fast more than safe, like a container. In this case I think that the overhead of a thread safe management is very small compared with the image loading and manipulation, so it’s a bad idea to use unguarded global variable in the library.

From DevIL site…

It is time for DevIL to get an updated interface, at least in my opinion. When I first started programming DevIL back in 2000, very few programs were multithreaded, and I was a really big fan of OpenGL, so that is why DevIL ended up with the interface it currently has. I am going to branch off the main trunk in Subversion to start DevIL version 2, which will be written internally in C++ but still have a C interface. DevIL 2 will also be thread-safe, which is one of the main goals of the rewrite. Finally, I will write a wrapper around DevIL 2 that will enable you to interface DevIL like it currently is, by using image names and binding images.

That was 2009, and the last update is from 2010. And that’s why now I use freeimage

Leonardi already described the point of multitreaded image manipulation. Thanks for that!

But I need to give you some more precise answers…

I get the point of usage the past tense. DevIL is not being updated for some time, but is certainly offers various types of image manipulation routines: loading, saving, compression, transformation (alienifyng, blurring contrast changing, equalizing,…), etc.

Currently just the two operations: decompression and re-compression.

[QUOTE=Alfonse Reinheart;1244371]
I don’t see how that makes it thread-safe. Wrapping DevIL calls in a singleton does nothing to stop you from calling into that singleton from multiple threads. [/QUOTE]
The mentioned singleton would be used as a context, and by using critical section would make each operation an atomic. But whatsoever, it is not a good idea since, as Leonardi said, it would serialize the whole process.

[QUOTE=Alfonse Reinheart;1244371]
A bigger question might be why you need multithreaded image loading.[/QUOTE]
Well, that’s a really bigger question. Uploading textures to a GPU is a very time consuming task. First, textures have to be read from the disk, or DVD, or downloaded form the net, which is the most time consuming operation (if we neglect caching for a while). Second, downloaded images are not GPU friendly. They are usually stored as JPEG or in some other format, which gives much better compression rate than DXT or other GPU-friendly schemes. Third, in order to increase access speed, there is some kind of RAM-cache. That cache should use some priority/LOD policy for the texture retrieval. All those tasks can be organized in a pipeline form to make use of multiple cores on a CPU and hide the latency. J.M.P van Waveren refers to it as “threaded pipeline” in his “Real-Time Texture Streaming & Decompression”. I had to extend his scheme by adding “an intelligent cache” with infinite number of priorities and self-organizing request queues. So I have a deeper pipeline. But, to make things worse, I have multiple independent data-sources. Making an intelligent cache for multiple sources is very hard to code. So, there are multiple pipelines (one for each storage/data_path). Of course, they are not all active at the same time, but nothing prevents simultaneous activity.

In such environment DevIL cannot be used. Yesterday I made a short overview of available image libraries, and it seams that std_image will be the best suite for jpg/png formats. Other will be treated differently. The only drawback is an inability to handle 16-bit-per-channel images. Luckily, they are rare.