16-bit float readback problem

Hello,

I made some texture loading / saving functions, and I have a problem with half precision floats (RGB16F ).

  1. When I create the data, I pass it to glTexImage as 32-bit floats as usual, so ok.

  2. When I save the data w/ glGetTexImage, I get widthheight3*2 bytes, which is ok ( ? )

  3. When I use later this data to create a texture, my texture is the half the size, which is natural, since glTexImage expects 32-bit float data and I feed it 16-bit. So I’ve got a problem here.

How can I make this work correctly? Is there any way for GL to convert the data I request to normal 32-bit floats? Or better, is there a way to feed to GL the 16-bit data & behave correctly?

Thanks in advance

glTexImage doesn’t expect anything except for what you’re telling it to expect. If your data is stored as 16bit fp, use GL_HALF_FLOAT_ARB instead of GL_FLOAT for the type argument of glTexImage2D.

N.

Thanks for your answer. I tried to load it as such but it messed up with the pixels,no luck. I checked up the dependencies in registry entry for ARB_texture_float, and GL_HALF_FLOAT_ARB is mentioned nowhere. Shouldn’t it be there??

It’s in the GL_ARB_half_float_pixel extension.

PS: When your pixels have a weird alignment try:
glPixelStorei(GL_PACK_ALIGNMENT,1);
glPixelStorei(GL_UNPACK_ALIGNMENT,1);

N.

Yes I have, you mean that if I didn’t, it would surely mess up? And about me telling to GL what to expect, GL only gets hints from the programmer for many things, including formats, correct me if I’m wrong…

[edit] That’s the way I have them set already, to be more specific, my texture init code is the following :
glPixelStorei (GL_UNPACK_ALIGNMENT , 1);
glPixelStorei (GL_UNPACK_SKIP_ROWS , 0);
glPixelStorei (GL_UNPACK_SKIP_PIXELS , 0);
glPixelStorei (GL_UNPACK_ROW_LENGTH , 0);
glPixelStorei (GL_UNPACK_SWAP_BYTES , 0);
glPixelStorei (GL_PACK_ALIGNMENT , 1);

The hints only apply for the internal formats, not for the type formats it reads from system memory.

N.

Can you provide some code? That would help a lot :slight_smile:

N.

It’s quite large just to see a particular bit, I’m actually making a texture superclass so I don’t think you’ll find it very useful, but anyway, here it goes…

I created the texture like this :


Texture * tex = new Texture2D(256,256,GL_RGB16F_ARB,GL_RGB,GL_HALF_FLOAT_ARB,vecdata,GL_TEXTURE_2D,"name",false,false);

which calls the ctor and does these :


_type = TEX2D;
	bind();
	_maxMiplevel = (loadAsRect) ? 0 : GLuint(texdata.size() - 1);

	// Check if compressed
	if((_internalFormat == GL_COMPRESSED_RGB_S3TC_DXT1_EXT) ||
		(_internalFormat == GL_COMPRESSED_RGBA_S3TC_DXT1_EXT) ||
		(_internalFormat == GL_COMPRESSED_RGBA_S3TC_DXT3_EXT) ||
		(_internalFormat == GL_COMPRESSED_RGBA_S3TC_DXT5_EXT))
	{
		_isCompressed = true;
		// Load all mipmap levels
		unsigned miplevel = 0;
		while(miplevel <= _maxMiplevel)
		{
			const unsigned char * tmp_data = texdata[miplevel].data;
			const double adjust = 1.0 / pow(2.0,int(miplevel));
			unsigned tmp_w = unsigned(_width * adjust),
					 tmp_h = unsigned(_height * adjust);
			glCompressedTexImage2D(_target,miplevel,_internalFormat,tmp_w,tmp_h,0,texdata[miplevel].size,tmp_data);
			++miplevel;
		}
		
	}
	else
	{
		_isCompressed = false;	
		// Load all mipmap levels
		if(genMipmaps)
			glTexParameteri(_target,GL_GENERATE_MIPMAP,GL_TRUE);
		unsigned miplevel = 0;
		while(miplevel <= _maxMiplevel)
		{
			const unsigned char * tmp_data = texdata[miplevel].data;
			const double adjust = 1.0 / pow(2.0,int(miplevel));
			unsigned tmp_w = unsigned(_width * adjust),
					 tmp_h = unsigned(_height * adjust);
			glTexImage2D(_target,miplevel,_internalFormat,tmp_w,tmp_h,0,_pixelFormat,_datatype,tmp_data);
			++miplevel;
		}
		glTexParameteri(_target,GL_GENERATE_MIPMAP,GL_FALSE);
	}
	if(loadAsRect)
	{
		setParam(GL_TEXTURE_WRAP_S,GL_CLAMP_TO_EDGE);
		setParam(GL_TEXTURE_WRAP_T,GL_CLAMP_TO_EDGE);
		setParam(GL_TEXTURE_MIN_FILTER,GL_NEAREST);	
		setParam(GL_TEXTURE_MAG_FILTER,GL_NEAREST);	
	}
	else
	{
		setParam(GL_TEXTURE_WRAP_S,GL_REPEAT);
		setParam(GL_TEXTURE_WRAP_T,GL_REPEAT);
		if(_maxMiplevel)
			setParam(GL_TEXTURE_MIN_FILTER,GL_LINEAR_MIPMAP_LINEAR);
		else
			setParam(GL_TEXTURE_MIN_FILTER,GL_LINEAR);
		setParam(GL_TEXTURE_MAG_FILTER,GL_LINEAR);	
	}

	// Compute bytes per texel

	GLint vals[7];
	glGetTexLevelParameteriv(_target,0,GL_TEXTURE_RED_SIZE,&vals[0]);
	glGetTexLevelParameteriv(_target,0,GL_TEXTURE_GREEN_SIZE,&vals[1]);
	glGetTexLevelParameteriv(_target,0,GL_TEXTURE_BLUE_SIZE,&vals[2]);
	glGetTexLevelParameteriv(_target,0,GL_TEXTURE_ALPHA_SIZE,&vals[3]);
	glGetTexLevelParameteriv(_target,0,GL_TEXTURE_LUMINANCE_SIZE,&vals[4]);
	glGetTexLevelParameteriv(_target,0,GL_TEXTURE_INTENSITY_SIZE,&vals[5]);
	glGetTexLevelParameteriv(_target,0,GL_TEXTURE_DEPTH_SIZE,&vals[6]);
	_bytesPerTexel += vals[0];
	_bytesPerTexel += vals[1];
	_bytesPerTexel += vals[2];
	_bytesPerTexel += vals[3];
	_bytesPerTexel += vals[4];
	_bytesPerTexel += vals[5];
	_bytesPerTexel += vals[6];
	_bytesPerTexel >>= 3;

I get the data size with this :


void Texture2D :: dlData(void * buffer,const GLuint miplevel) const
{
	bind();
	if(_isCompressed)
	{
		glGetCompressedTexImage(_target,miplevel,buffer);
	}
	else
		glGetTexImage(_target,miplevel,_pixelFormat,_datatype,buffer);
}

And I save it like this :


bool TextureIO :: _saveTEX(const Texture * tex, 
						  const std::string& fname)
{
	TEX_header header = tex->getTEXheader();
	
	FILE * fp = fopen(fname.c_str(),"wb");
	if( fp == NULL )
	{
	   CKLOG(string("Error opening file ") + fname,0);
       return 0;
	}

	// Write header
	fwrite(&header,sizeof(header),1,fp);

	if(!_saveMipmaps)
		header.mipcount = 1;

	unsigned faces = (header.textype == TEXCUBE) ? 6 : 1;
	for(unsigned i=0;i<faces;++i)
	{
		for(unsigned j=0;j<header.mipcount;++j)
		{
			// write mipmap level
			unsigned size = tex->dataSize(j);
			unsigned char * data = MemMgrRaw::instance()->allocate<unsigned char>(size);
			if(header.textype == TEXCUBE)
				tex->dlData(data,CubemapFace(i),j);
			else
				tex->dlData(data,j);
			fwrite(&size,sizeof(unsigned),1,fp);
			fwrite(data,sizeof(unsigned char),size,fp);
			MemMgrRaw::instance()->free<unsigned char>(data);
		}
	}
	fclose(fp);

	return true;
}

And I load it like this :


Texture * TextureIO :: _loadTEX(const std::string& fname)
{
	// Log Event : Not Implemented Yet!
	FILE * fp = fopen(fname.c_str(),"rb");
	if( fp == NULL )
	{
	   CKLOG(string("Error opening file ") + fname,0);
       return 0;
	}
	TEX_header header;
	size_t read = fread(&header,sizeof(header),1,fp);
	if((!read) || (header.magic != 1234))
	{
	   CKLOG(string("Invalid TEX header in file ") + fname,0);
       return 0;
	}
	vector<vector<MipmapLevel> > data;
	
	unsigned faces = (header.textype == TEXCUBE) ? 6 : 1;
	if(header.textype != TEX3D)
		header.depth = 1;

	// Get the size of mip 0 of the tex (or 1 of 6 in cubemaps)
	unsigned size(0);
	// Get the data
	for(unsigned i=0;i<faces;++i)
	{
		vector<MipmapLevel> single_data;
		for(unsigned j=0;j<header.mipcount;++j)
		{
			fread(&size,sizeof(unsigned),1,fp);
			MipmapLevel miplevel(MemMgrRaw::instance()->allocate<unsigned char>(size),size);
			fread(miplevel.data,1,size,fp);
			if(_genMipmaps || (!j))
				single_data.push_back(miplevel);
			else
				MemMgrRaw::instance()->free<unsigned char>(miplevel.data);
		}
		data.push_back(single_data);
	}
	fclose(fp);

	header.mipcount = 1;

	Texture * tex(NULL);
	bool genMips = (header.mipcount > 1) ? false : _genMipmaps;

	// Create the textures
	switch(header.textype)
	{
		case TEX1D:
			break;
		case TEX2D :
			{
			bool loadrect = (header.compressed || _genMipmaps) ? false : _loadAsRect;
			unsigned target = loadrect ? GL_TEXTURE_RECTANGLE_ARB : GL_TEXTURE_2D;
			tex = new Texture2D(header.width,header.height,header.ifmt,header.pfmt,
								header.datatype,data[0],target,truncDir(fname),genMips,loadrect);
			}
			break;
		case TEX3D :
			tex = new Texture3D(header.width,header.height,header.depth,header.ifmt,header.pfmt,
								header.datatype,data[0],GL_TEXTURE_3D,truncDir(fname),genMips);
			break;
		case TEXCUBE :
			tex = new TextureCube(header.width,header.height,header.ifmt,header.pfmt,
								header.datatype,data,GL_TEXTURE_CUBE_MAP,truncDir(fname),genMips);
			break;
		default :
			assert(0);
	}

	// Release memory
	for(unsigned i=0;i<faces;++i)
		for(unsigned j=0;j<header.mipcount;++j)
			MemMgrRaw::instance()->free<unsigned char>(data[i][j].data);

	return tex;
}

I don’t expect of you to read all this stuff, but you asked for it :slight_smile:

Just a quick question… are you experiencing the same problem for GL_RGB32F?

N.

nope, works like a charm ( that is, IF I use GL_NEAREST & not GL_LINEAR for filters, else it sloooooooooows dooooooooooooown)

Maybe it’s best to write a small example exhibiting the problem:

data -> glTexImage2D
glGetTexImage -> newdata
newdata -> glTexImage2D

N.

I’ll try that & post later my results, thanks again!

Well, here is the used code & my observations along with it :


	glEnable(GL_TEXTURE_2D);
	// Create texture
	glGenTextures(1,&tex);
	glBindTexture(GL_TEXTURE_2D,tex);
	float * dataptr = MemMgrRaw::instance()->allocate<float>(256*256*3);
	// Fill float data
	{
		for(int i=0;i<256;++i)
			for(int j=0;j<256;++j)
			{
				const int index = (i*256 + j)*3;
				dataptr[index] = i / 255.0f;
				dataptr[index+1] = j / 255.0f;
				dataptr[index+2] = 0.5f;
			}
	}
	// Create tex image - upload data
	glTexImage2D(GL_TEXTURE_2D,0,GL_RGB16F_ARB,256,256,0,GL_RGB,GL_FLOAT,dataptr);
	glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_NEAREST);

	// set data to 0
	memset(dataptr,0,sizeof(float)*256*256*3);
	// download data
	glGetTexImage(GL_TEXTURE_2D,0,GL_RGB,GL_HALF_FLOAT_ARB,dataptr);
	// delete texture
	glDeleteTextures(1,&tex);

	// create a new texture
	glGenTextures(1,&tex);
	glBindTexture(GL_TEXTURE_2D,tex);
	// create tex image - upload downloaded data
	glTexImage2D(GL_TEXTURE_2D,0,GL_RGB16F_ARB,256,256,0,GL_RGB,GL_HALF_FLOAT_ARB,dataptr);
	glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_NEAREST);

	MemMgrRaw::instance()->free<float>(dataptr);

(I use halffloat-halffloat for the example)

When downloading using half float & uploading again using half float, it’s ok

When downloading using float & uploading again using float, it’s ok

I use normally for this case GL_FLOAT to download, but half the total byte size (because of the values I query from GL). So
I should have exactly the half of the image, from bottom to top. But I don’t, I have the whole image, ‘compressed’ into the lower half (possibly by reading 1 for each two values). So I’m still confused…

OK. So now you’ve posted the code that works. How about posting the code that doesn’t work? :wink:

N.


glEnable(GL_TEXTURE_2D);
	// Create texture
	glGenTextures(1,&tex);
	glBindTexture(GL_TEXTURE_2D,tex);
	float * dataptr = MemMgrRaw::instance()->allocate<float>(256*256*3);
	// Fill float data
	{
		for(int i=0;i<256;++i)
			for(int j=0;j<256;++j)
			{
				const int index = (i*256 + j)*3;
				dataptr[index] = i / 255.0f;
				dataptr[index+1] = j / 255.0f;
				dataptr[index+2] = 0.5f;
			}
	}
	// Create tex image - upload data
	glTexImage2D(GL_TEXTURE_2D,0,GL_RGB16F_ARB,256,256,0,GL_RGB,GL_FLOAT,dataptr);
	glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_NEAREST);

	// set data to 0
	memset(dataptr,0,sizeof(float)*256*256*3);
	// download data
	glGetTexImage(GL_TEXTURE_2D,0,GL_RGB,GL_FLOAT,dataptr);
	// delete texture
	glDeleteTextures(1,&tex);

// ** I effectively do this when I save to disk & load again **
	const unsigned hfsize = (sizeof(float)*256*256*3)/2;
	memset(&dataptr[(hfsize)/4],0,hfsize);
// **

	// create a new texture
	glGenTextures(1,&tex);
	glBindTexture(GL_TEXTURE_2D,tex);
	// create tex image - upload downloaded data
	glTexImage2D(GL_TEXTURE_2D,0,GL_RGB16F_ARB,256,256,0,GL_RGB,GL_FLOAT,dataptr);
	glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_NEAREST);

	MemMgrRaw::instance()->free<float>(dataptr);

Well, here’s the code that emulates my problem, I only write to disk half of the data (I set it here to zero with memset) since I guessed I got them as 16-bit data. So finally I found what’s wrong, I get half of my data. So, sincerely MANY THANKS for your time… One side question though, if I always use GL_HALF_FLOAT_ARB for my 16-bit data, will I have any problems, performance wise or otherwise??

That depends. If you’re just using half floats to transfer data from the GPU and then use the same data to upload it again, there’s no performance loss. In fact, if half float is the accuracy you need, it’s faster than 32bit float. However, if you’re planning to perform some operations on the CPU with half float data you will get a performance hit because CPU’s are designed to work on 32 bit floats, not half floats.

I still don’t get why you want to save the data to 32 bit floats on the CPU, then convert it to half float and upload it again. I fail to see the point of this.

In normal cases you have to make sure that the next to last parameter to glTexImage2D (type) correponds to the data type of the pointer you’re providing. If your data is stored as floats and you use a type of half float, it gets messed up. Vice versa, if your data is stored as half floats and you use a type of float it also gets messed up. GL takes care of the conversion of your type to the internal format.

e.g. you have a float buffer with all values set to 0.5f. You upload it using glTexImage with internal format GL_RGB_32F with type GL_FLOAT and get the values using glGetTexImage with type GL_FLOAT into a float buffer. You will get back values of 0.5f.

if you have a float buffer with all values set to 0.5f. You upload it using glTexImage with internal format GL_RGB_16F with type GL_FLOAT and get the values using glGetTexImage with type GL_FLOAT into a float buffer. You will get back values of 0.49… because the 32bit 0.5f value has been converted to 16bit with glTexImage and back to 32bit because you specified GL_FLOAT with glGetTexImage.

N.

I’ll actually intend to use 16-bit floats only in GPU computations, the reason I want the save / load is for snapshots of float data (export), & viewing them in an external viewer(import) I made. Thanks for all the clarifications by the way, I used blindly GL_FLOAT all the time, although I had 16-bit internal formats sometimes, so this post - problem - solution made me realize some things :slight_smile:

What I meant about performance-wise or problem-wise is this :
Considering these two calls, what’s the difference ?? how will the data be stored inside ?? 16 or 32 bit?? because I thought it would be 16 bit & I’m just starting to shiver (after a deferred shading implementation with 3 of RGBAF_16 buffers & GL_FLOAT) …

glTexImage2D(GL_TEXTURE_2D,0,GL_RGBF16_ARB,…,GL_FLOAT,NULL)
glTexImage2D(GL_TEXTURE_2D,0,GL_RGBF16_ARB,…,GL_HALF_FLOAT_ARB,NULL)

Both calls generate exactly the same internal format on the GPU.

These calls:

glTexImage2D(GL_TEXTURE_2D,0,GL_RGBF16_ARB,…,GL_FLOAT,floatdata)
glTexImage2D(GL_TEXTURE_2D,0,GL_RGBF16_ARB,…,GL_HALF_FLOAT_ARB,halffloatdata)

also result in exactly the same image data internally given that the halffloatdata is a conversion of the floatdata following the specs of the GL_ARB_half_float_pixel extension.

Note that not all cards support 16 bit natively, so it’s possible that it uses GL_RGBF32_ARB internally instead of GL_RGBF16_ARB.

N.

Ok, I’ll be using them a lot I guess, and the knowledge of their exact differences will surely help a lot. So, thanks again!

First of all hello to everyone. At last I think I have found a place to share my questions :slight_smile:

I do have a similar problem. First of all when I use GL_RGBA16F_ARB, what should be the size of the dynamic memory I have taken? WidthHeightsizeof(float)*4?

Right now I am using GL_RGBA16F_ARB. I use FBO to put the results in the texture created after the computation in GPU is done and I do read it back using getTextureData. But when I just resend it using:

glBindTexture(m_target, m_texID);
glTexSubImage2D(m_target, 0, 0, 0, m_w, m_h, m_format, GL_FLOAT, m_data);
glBindTexture(m_target, 0);

and display the texture on the screen it is like everything is a bit mixed up. I tried using both GL_HALF_FLOAT_ARB and GL_FLOAT for the format but it made no difference.

If I display the texture before the round trip everything is fine.

Anyone have any clue?