View Full Version : Loss of precision

castellp

03-09-2006, 04:06 AM

Hello, I'm doing a render to texture (GL_RGBA16). And in my GLSL fragment program I do: gl_FragColor = vec4(float(n) / 65536.0, 0.0, 0.0, 0.0). n is a variable I compute inside the shader.

Later, I do a readPixels() and then every pixel is multiplied by 65536.0, so I will get n. But I get different results, if I do the same and use 256.0.

Why?

Any idea? I need to use 16 bits per channel. My n variable has a value within 0 and 65536.

Thank you,

Pascual.

killerseven

03-09-2006, 11:24 AM

You cannot represent the floating point value 65536.0 exactly with 16-bits, so you will see differences.

castellp

03-10-2006, 01:01 AM

First, thank you for your answer.

My n variable represents de projected area of a triangle in pixels. Due to the limitations of the shaders programs, I'm doing several passes. In each pass, I get one row of the framebuffer. So n will be 256 for a row (as a maximum). At the end, I have to sum up all the partial areas. Then, I'm using the blend function. For a 256x256 image, my n variable could be as a maximum 65536.0

My idea is to divide the n value in the shader and finally multiple the value in my program. As I said before.

Any alternative?

Thank you.

Relic

03-10-2006, 02:40 AM

The 16 bit floating point format is sign/exp/mantissa: s1e5m10

You cannot get more precise with this than 10 bits in the range of 0.0 and 1.0.

What exactly do you want to achieve? There might be simpler ways.

killerseven

03-10-2006, 04:46 PM

I'm not sure what the purpose of your shader is, but if you are simply trying to calculate the number of pixels written when rendering a triangle, it would be much simpler and far more efficient to use occlusion querying.

castellp

03-12-2006, 11:57 PM

Thank you all, for your answers. I want to compute the projected area of a triangle in pixels. Every triangle is rendered with a different colour and the shader is going to analized each row. So, in order to obtain the final result I need to sum up this partial results. Due to the HW limitations, I'm using a 256x256 image. Thus, these partial areas will be 256 as a maximum (on each row) and the maximum projected area for this image will be 65536.

My idea was to divide this partial areas by 65536 and use the blend fucntion tu sum up this partial results. The final result will be multiplied by 65536 at the end. But I need more precision to do this.

I guess there might be other alternatives but I don't know them.

Is is possible to do this?

Pascual.

ZbuffeR

03-13-2006, 01:53 AM

as said killerseven, occlusion queries looks like the best tool:

http://oss.sgi.com/projects/ogl-sample/registry/ARB/occlusion_query.txt

castellp

03-13-2006, 04:00 AM

I've tried with occlusion query but it's too slow, so that's why I decided to do it using the GPU.

killerseven

03-13-2006, 11:15 AM

Occlusion querrying is only slow if you are stalling it to get the result of the current frame.

If you create 3 or 4 queries a frame apart from each other, you should be able to query the result of frame 1 during frame 3 or 4 at little to no cost.

Usually getting a fractionally delayed result is not a problem, I guess it depends why you need the area of the triangle.

k_szczech

03-14-2006, 09:07 AM

"I want to compute the projected area of a triangle in pixels"

Does it mean that you're not interested in the amount of pixels visible on the screen but you simply want to calculate the size of this triangle as if it would be the only one?

If Yes, then don't bother with rendering to texture and using shaders - there is a simple and fast solution for it using feedback mode. Let me know if that's what you are interested in.

castellp

03-15-2006, 02:08 AM

Originally posted by k_szczech:

"I want to compute the projected area of a triangle in pixels"

Does it mean that you're not interested in the amount of pixels visible on the screen but you simply want to calculate the size of this triangle as if it would be the only one?

If Yes, then don't bother with rendering to texture and using shaders - there is a simple and fast solution for it using feedback mode. Let me know if that's what you are interested in. Well, really I want to compute the projected area of a visible triangle in pixels. I'm sorry if I didn't express myself correctly.

Powered by vBulletin® Version 4.2.3 Copyright © 2017 vBulletin Solutions, Inc. All rights reserved.