PDA

View Full Version : 'W buffering" with opengl, how to?



michagl
03-23-2005, 12:45 PM
i get the impression that i should be utilizing 'W buffering' as opposed to z buffering. i found some directx initialization code, but haven't been able to find anything for opengl. i get the impression it is a commonly supported feature though.

i'm also curious if anyone knows of a list of nvidia cards and their features. i'm mostly looking for a z depth precision comparison between cards.

are cards with 64bit depth buffers ultra expensive? is the extra precision really worth it given exponential scaling of pseudodepth? would W buffering perhaps be a more lucrative use of the extra precision.

finally is there any unecesarry performance cost in using W buffering on typical hardware?

sincerely,

michael

dorbie
03-23-2005, 01:17 PM
From hints given in personal correspondance hardware implementations exploit the screenspace 2D interpolation linearity of a zbuffer (the corollary of non linearity w.r.t. true eye Z) for features like coarse z buffer optimizations.

So the costs aren't fully clear anymore, there was a time when folks suggested (I think mainly at 3Dfx) that hardware was up to task of using perspective correction for pathological choices for near and far clip. Now that may not be the case because it complicates popular optimizations like coarse z.

I like the non linearity of a zbuffer for reasonable clip plane values, it really only sucks when you crank the near plane in too far. There are other schemes besides w buffering that try to correct for this too.

michagl
03-23-2005, 01:56 PM
Originally posted by dorbie:
From hints given in personal correspondance hardware implementations exploit the screenspace 2D interpolation linearity of a zbuffer (caused by non linearity w.r.t. true eye Z) for features like coarse z buffer optimizations.

So the costs aren't fully clear anymore, there was a time when folks suggested (I think mainly at 3Dfx) that hardware was up to task of using perspective correction for pathological choices for near and far clip. Now that may not be the case because it complicates popular optimizations like coarse z.

I like the non linearity of a zbuffer for reasonable clip plane values, it really only sucks when you crank the near plane in too far. There are other schemes besides w buffering that try to correct for this too.so i take it that w buffering is not a commonly supported feature, even though D3D seems to have a built in functionality to facilitate it?

i read something before about W buffering in another forum here. i will probably try to find it, even though it sounds like a lost cause. it seems like popular games utilize it, but they may've been games from software rendering days.

my basic problem occurs not when the near plane is too small, but when the near per far ratio is too small. so like 1/100 is the same as 10/1000.

i'm working at the scale of a small planet, only the planet is cylindrical, so that the horizon curves up, meaning you don't get the benefit of distant lands being lost below the horizon with curvature.

basicly the near plane is at which point the camera clips through the space... so it is the only non relative judgement of scale.

i really need to drop the size of the near plane or increase the size of the environment so that the camera can go down to a human scale.

around a 0.1 type near plane swimming seems to occur. but if you push the near and far up and the same ratio, the swimming goes away, but there is a pretty well defined point were depth values begin to merge and then actually...

i'm actually seeing a weird effect i don't understand. it must have something to do with the z buffer though. basicly imagine rendering two cylinders one nested inside the other. the diameters are relatively close... one is actually terrain, then another a layer of atmosphere.

the atmosphere is the inside cylinder. the camera is inside the cylinders. now the crazy thing happening is the further away the surfaces are along the z axis. the inner cylinder begins to intersect the outer cylinder, and then actually goes behind the outer cylinder.

how is this possible?

it has nothing to do with the modelview matrix... if you move to the point were the cylinders invert then the effect moves.

i really don't understand how say two identical geometries, one scaled slightly smaller, can totally intersect until the smaller is totally outside the larger increasingly in the z axis?

any thoughts?



From hints given in personal correspondance hardware implementations exploit the screenspace 2D interpolation linearity of a zbuffer (caused by non linearity w.r.t. true eye Z) for features like coarse z buffer optimizations.
oh, so w buffering can't be linearly interpolated in screen space? still by this day and age, it seems like dedicated hardware doing a per pixel depth calculation could be fit in somewhere couldn't it?

just to show i don't get out much... what does w.r.t. mean btw?

dorbie
03-23-2005, 03:07 PM
It means "with respect to".

I don't know what's supported out there, OpenGL intentionally isn't explicit about what's in the depth buffer or the system used, only what you get when you read it. Some systems have used pseudo float to adjust the precision of stored values for example but have hidden this from the observer.

There's also this extension:

http://oss.sgi.com/projects/ogl-sample/registry/EXT/wgl_depth_float.txt

Although the preamble is somewhat bogus because it ignores scale and the resulting ramp somewhat arbirtary. This should really be to compensate for non linearity of z by ramping a float storage running in the opposite direction. The idea of using fixed point in there is just bizarre because it makes the extension do nothing. I assume (for the sake of my own sanity) that a fixed point implementation would have higher precision fixed point interpolators with a cast to float at lower precision prior to storage.

michagl
03-23-2005, 04:48 PM
Originally posted by dorbie:
It means "with respect to".

I don't know what's supported out there, OpenGL intentionally isn't explicit about what's in the depth buffer or the system used, only what you get when you read it. Some systems have used pseudo float to adjust the precision of stored values for example but have hidden this from the observer.

There's also this extension:

http://oss.sgi.com/projects/ogl-sample/registry/EXT/wgl_depth_float.txt

Although the preamble is somewhat bogus because it ignores scale and the resulting ramp somewhat arbirtary. This should really be to compensate for non linearity of z by ramping a float storage running in the opposite direction. The idea of using fixed point in there is just bizarre because it makes the extension do nothing. I assume (for the sake of my own sanity) that a fixed point implementation would have higher precision fixed point interpolators with a cast to float at lower precision prior to storage.yeah, i don't follow that extension either. the way it is written is insane... maybe its a practical joke or something? someone maybe just made a bet about whether they could get microsoft to adopt a nonsensical opengl extension?

as for my problem as far as consequentiality is concerned... sorry false alarm. i finally found a case that i could not rationalize. so i started looking through my code for anything. i actually was very lucky that i found it. it was one of those bugs that is so far out that you never could've believed it could go on unoticed under equally far out circumstances. the route of it all was a flag had a not operator that got stuck in front of it, and reverted to an alternative rendering mode i had done some work with to improve spatial precision at insanely high scales. it works like a charm, only flaw is in the depthbuffer it produces, which can't be rectified with opengl so far as far as i know. (it needs some way to scale depth values) the fact that it worked so well under recent circumstances was shocking.

so everything is back to normal now. i'm just glad i only lost an entire day to this. finding the bug was shear luck.

never the less, thanks for the input dorbie.

michagl
03-23-2005, 04:59 PM
well actually... that explained why the closer surface was disappearing behind the further surface.

but as i expected the general precision and swimming problem is still there. i was hoping the problem would've magicly cleared up after fixing that bug.

but trying to get to get the scale down to an acceptable human scale still produces the same issues.

that is why really it took me so long to come around to the bug... because i was expecting this issue to begin with.

so i'm still very open to discussion.

especially interested in a list of nvidia card features.

i'm not very good with internet queries i guess. i can never find anything on the internet.

dorbie
03-23-2005, 06:09 PM
yeah, i don't follow that extension either. the way it is written is insane... maybe its a practical joke or something? someone maybe just made a bet about whether they could get microsoft to adopt a nonsensical opengl extension?
Not really, in practice I think the intention is to implement an floating point numerical precision ramp (approximately log2 ?) with high precision at the far plane due to the lower exponent (that's why the z value is reversed) and less precision at the near as the exponent increases to compensate for the hyperbolic perspective z ramp from near to far that's determined by the near and far clip plane locations. It's just that some of the discussion in the extension text is utter nonsense, the idea itself makes some sense if you could implement it efficiently and you chose near and far values where the extension didn't hose your near plane precision.

michagl
03-23-2005, 07:51 PM
Originally posted by dorbie:

yeah, i don't follow that extension either. the way it is written is insane... maybe its a practical joke or something? someone maybe just made a bet about whether they could get microsoft to adopt a nonsensical opengl extension?
Not really, in practice I think the intention is to implement an floating point numerical precision ramp (approximately log2 ?) with high precision at the far plane due to the lower exponent (that's why the z value is reversed) and less precision at the near as the exponent increases to compensate for the hyperbolic perspective z ramp from near to far that's determined by the near and far clip plane locations. It's just that some of the discussion in the extension text is utter nonsense, the idea itself makes some sense if you could implement it efficiently and you chose near and far values where the extension didn't hose your near plane precision.yeah i was just kidding naturaly. it does seem like something that would never be implimentable in hardware though unless there was some custom piece of hardware being used with some group with decent clout. i really don't know what i'm talking about though. it just doesn't look promising though... maybe it does work, maybe that is how directx gets a w buffer api. i might just try it to see what it does. i figure it will do nothing, that is not take.

will depth buffers be 64bit standard in near future. in my experience it isn't really possible to render an outdoor scene with a realistic horizon and a camera near enough to some geometry in the foreground not to clip it to death... not even close actually. colour buffers don't need depth. but i wouldn't mind max 128bits of depth standard.

how can you find out how many depth bits a card will support? just for the sake of shopping that is. i'm thinking i want at least 64bits on my next card.

is the linear interpolation through screen space accurate numericly? or is it just a precision issue that gives the buzzsaw effect? would per pixel depth computation with the standard algorithm yield less swimming?

michagl
03-23-2005, 08:01 PM
i have to get to sleep.

real quick the extension requires:

wglChoosePixelFormatExEXT

but if you put that into google you get nothing useful.

is this microsoft internal stuff? or something brand new? beats me.

edit: i can't get it with wglGetProcAddress

michagl
03-24-2005, 09:12 AM
according to the win2k API's ChoosePixelFormat function, my card supports a 128bit depth buffer.

the msvc docs say that this function chould change the passed pixel descriptor's attributes to the closest support match. if i give it, 128 it gives me back 128, same for 32 and 64... i can't discern any visual improvement though.

at scales i like, i get very noticible swimming in the depth buffer when the modelview matrix is rotating. if it is just translating there is no swimming.

edit: for all i know my current card (a low end QuadroFX model) might only do a 16bit depth buffer.

zed
03-24-2005, 12:26 PM
according to the win2k API's ChoosePixelFormat function, my card supports a 128bit depth bufferi know of no card that has 128bit depth
all nvidia cards only support 16 or 24 (no 32) please correct me if im wrong.
its important to query how many bits u get given and not assume u get what u asked for.
FWIW with nvidia cards if u ask for 32 u will get back 16 (and not 24 which is closer)

al_bob
03-24-2005, 01:16 PM
the msvc docs say that this function chould change the passed pixel descriptor's attributes to the closest support match. if i give it, 128 it gives me back 128, same for 32 and 64... i can't discern any visual improvement though.No it doesn't.

Behold:


MSDN:

int ChoosePixelFormat(
HDC hdc, // device context to search for a best pixel format
// match
CONST PIXELFORMATDESCRIPTOR * ppfd
// pixel format for which a best match is sought
);The pixel format descriptor is const.

What you actually need to do is check the return value, and if it's valid, call DescribePixelFormat() with that pixel format.

Edit: Details can be found MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/opengl/ntopnglr_2qb8.asp)

michagl
03-24-2005, 01:22 PM
Originally posted by zed:

according to the win2k API's ChoosePixelFormat function, my card supports a 128bit depth bufferi know of no card that has 128bit depth
all nvidia cards only support 16 or 24 (no 32) please correct me if im wrong.
its important to query how many bits u get given and not assume u get what u asked for.
FWIW with nvidia cards if u ask for 32 u will get back 16 (and not 24 which is closer)thanks for the info. if that is true there is probably a lot of misinformation out on the internet regarding such matters.

i tried setting my pixel format to 24bits, but i can't see any visual improvement so i assume it didn't take.

the microsoft api is passed a pixel descriptor structure by address... it is supposed to modify the attributes to the closest support match according to the docs... but since when can you expect microsoft to live up its word. i have no idea what goes on behind the scenes though, maybe there is no way to validate the supported bits... though i assume there must be.

how can you find out what cards have what bits available? stencil buffer is shared with depth bits right? i turned off stencil bits.... is there anything else i can do to try to increase depth precision?

and again, i how can you find out what cards support what with the depth buffer... that will be near the top of my list when i start looking at cards in the future.

edit: just for the record, i asked for 32 and got back 24. i could use a lot more though it seems.

michagl
03-24-2005, 01:31 PM
[QUOTE]
Behold:
[QUOTE]

yeah, i didn't pay enough attention i guess.

in my local docs there are remarks that say "the function returns a pixel format with [an adjusted value]... but i guess this is referring to the handle returned. it looks like DescribePixelFormat can retrieve the actual results from the handle.

thanks for pointing that out.

knackered
03-24-2005, 03:02 PM
To get around the precision problem you're having, may I recommend you render your scene in slices along the z axis (in back to front order), specifying a new frustum and clearing the depth buffer before each new slice. Play around to find the optimal number of slices for the depth range of your scene versus your zbuffer precision.

knackered
03-24-2005, 03:20 PM
This is the general procedure for setting up your pixel format with the unextended pixel format functions:-

HGLRC createOpenGLContext(HDC dc, int& colourBits, int& depthBits, int& stencilBits, bool& stereo)
{
PIXELFORMATDESCRIPTOR requested_pfd =
{sizeof(PIXELFORMATDESCRIPTOR), 1, 0, PFD_TYPE_RGBA, colourBits, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, depthBits, stencilBits, 0, PFD_MAIN_PLANE, 0, 0, 0, 0};

requested_pfd.dwFlags = PFD_DRAW_TO_WINDOW | PFD_SUPPORT_OPENGL | PFD_DOUBLEBUFFER;
if (stereo) requested_pfd.dwFlags |= PFD_STEREO;

// get a pixel format index for a format that closely matches 'requested_pfd'

int pixelFormatIndex = ::ChoosePixelFormat(hDC, &requested_pfd);

if (!pixelFormatIndex)
{
fprintf(stderr, "Unsupported context attributes.\n");
return 0;
}

PIXELFORMATDESCRIPTOR chosen_pfd;

// now get the attributes of the pixel format that was chosen for us

if (!::DescribePixelFormat( hDC,
pixelFormatIndex,
sizeof(chosen_pfd),
&chosen_pfd))
return 0;

// compare our desired pixel format with the one given to us...

if (chosen_pfd.cColorBits != requested_pfd.cColorBits)
{
fprintf(stderr, "Unsupported colour depth of %i. Max allowed=%i\n", colourBits, chosen_pfd.cColorBits);
colourBits = chosen_pfd.cColorBits;
}

if (chosen_pfd.cDepthBits != requested_pfd.cDepthBits)
{
fprintf(stderr, "Unsupported z depth of %i. Max allowed=%i\n", depthBits, chosen_pfd.cDepthBits);
depthBits = chosen_pfd.cDepthBits;
}

if (chosen_pfd.cStencilBits != requested_pfd.cStencilBits)
{
fprintf(stderr, "Unsupported stencil depth of %i. Max allowed=%i\n", stencilBits, chosen_pfd.cStencilBits);
stencilBits = chosen_pfd.cStencilBits;
}

if (stereo && !(chosen_pfd.dwFlags & PFD_STEREO))
{
fprintf(stderr, "Unable to support Stereo!\n");
stereo = false;
}

// now actually set the pixel format of the device context to the one chosen for us
if (!::SetPixelFormat(hDC, pixelFormatIndex, &chosen_pfd))
{
fprintf(stderr, "Unsupported context attributes.\n");
return 0;
}

// create the OpenGL context from our newly modified device context

return ::wglCreateContext(hDC);
}

Obli
03-25-2005, 10:24 AM
Originally posted by michagl:

Originally posted by zed:
[QUOTE]...with nvidia cards if u ask for 32 u will get back 16 (and not 24 which is closer)...
edit: just for the record, i asked for 32 and got back 24. i could use a lot more though it seems.I just wanted to say that this behaviour is probably setting-dependant or driver-dependant.
I once asked for a 32bit Z and they didn't work (witout stencil), there were no pixelformat avaiable.
So, this fallback is probably a not-so-smart driver move, recognizing you're doing something messy.

michagl
03-25-2005, 12:14 PM
Originally posted by knackered:
To get around the precision problem you're having, may I recommend you render your scene in slices along the z axis (in back to front order), specifying a new frustum and clearing the depth buffer before each new slice. Play around to find the optimal number of slices for the depth range of your scene versus your zbuffer precision.under my circumstances there is no real obvious way to do that... the geometry is all continuous.

the main problem i'm having is with distant innerpenetrating geometry. for instance, a mountain interpenetrating a cloud layer. the clouds just go through the mountains, and there is no realistic way to get around that. so you have to rely on the z buffer to work out the clipping.

at a fair distance, the clipping 'swims' significantly based on the rotational components of the modelview matrix.

if someone can offer an optimal environment sale and near/far plane settings to fully utilize the z buffer that might be helpful... but from my run-time modulatable expiriments it seems i'm already doing that.

i realize my needs are not comformant with the game industry right now. i've never seen games with realistic expansive environments. i'm thinking about using a per pixel fragment shader to manually set the depth fragments. if there is a greater cost to this than simply the necesarry computations and the loss of the early z-test please let me know.

there has been a discussion on methods somewhere around here. i think it was in the opengl wish list forum. i plan to look it up as soon as i get a chance.

Aeluned
03-25-2005, 01:55 PM
i'm thinking about using a per pixel fragment shader to manually set the depth fragments.
You can't alter the depth of fragments in a fragment shader.

dorbie
03-25-2005, 03:02 PM
Yes you can!

knackered
03-25-2005, 03:50 PM
michagl, you must have a broad way of rendering everything in the background and everything in the foreground separately. That's all it takes, as you need to concentrate a large range of the zbuffer in the foreground only when rendering the foreground.
Basically, setup a frustum with the near plane being at, say 100, and the far plane at 10000 for the 1st slice, draw the scene, clear the zbuffer, then set up the frustum with the near plane at 0.1 and the far plane at 100 for the next slice, draw the scene and swapbuffers.
The standard view frustum hardware clipping will 'stitch' the two slices together seemlessly.
Your frustum culling code will optimise away any unnecessary work...but if you haven't got any culling method, it won't affect the visual results, only performance.

michagl
03-25-2005, 05:40 PM
Originally posted by knackered:
michagl, you must have a broad way of rendering everything in the background and everything in the foreground separately. That's all it takes, as you need to concentrate a large range of the zbuffer in the foreground only when rendering the foreground.
Basically, setup a frustum with the near plane being at, say 100, and the far plane at 10000 for the 1st slice, draw the scene, clear the zbuffer, then set up the frustum with the near plane at 0.1 and the far plane at 100 for the next slice, draw the scene and swapbuffers.
The standard view frustum hardware clipping will 'stitch' the two slices together seemlessly.
Your frustum culling code will optimise away any unnecessary work...but if you haven't got any culling method, it won't affect the visual results, only performance.there is no foreground and background... its a steady amalgom of terrain, clouds, water, and eventually other natural artifacts and inhabitants all the way from the front to the back. the whole scene is pushed through clod so that the background can be insanely far away without running into traditional crowding at a distance.

i can't use flood fogging, because the nature of the world i'm rendering is in its untraditional makeup. its in a cylinder, the horizon curves up, and there are cables here and there holding the whole world together against centripetal gravity. fogging it out to infinity defeats the whole point. the air is generally said to be prestine as well so that you can see the furthest lands until they disappear behind the 180 degree upward curving horizon.

i'm pretty certain i'm going to go for per pixel depth fragments. the only caveats remaining, is is it legal to use a pixel shader without a vertex shader bound, and if so, what should i expect as inputs. the way cg is setup, it seems i can declare the inputs i'm interest in the fragment shaders argument list. but i'm not sure what i should do exactly. am i going to have to create a fragment shader for every possible rendering state? should i just do frag shaders for the optimal 'path' rendering states? and should i try to do a general purpose custom frag shader that can somehow accept most any rendering state and all inputs? should i just give up on rendering debug lines for instance to the depth buffer? or can i somehow produce a shader which can accomodate a large swath of general purpose rendering states? does the hardware change the static pipeline shader as its rendering states change? etc etc... i have a feeling what i will end up doing is just revert to the junk depth buffer routine when not rendering through the optimal 'path'.

michagl
03-25-2005, 06:59 PM
Originally posted by knackered:
michagl, you must have a broad way of rendering everything in the background and everything in the foreground separately. That's all it takes, as you need to concentrate a large range of the zbuffer in the foreground only when rendering the foreground.
Basically, setup a frustum with the near plane being at, say 100, and the far plane at 10000 for the 1st slice, draw the scene, clear the zbuffer, then set up the frustum with the near plane at 0.1 and the far plane at 100 for the next slice, draw the scene and swapbuffers.
The standard view frustum hardware clipping will 'stitch' the two slices together seemlessly.
Your frustum culling code will optimise away any unnecessary work...but if you haven't got any culling method, it won't affect the visual results, only performance.i gave this some more thought, and i think there may be something to it worth implimenting if the numbers work out ok.

my environment is partitioned pretty well. it will be very tricky to impliment, but basicly i think if i was two keep three different back planes set by the user. every node in the partition that falls behind the front back plane could be drawn. then the depthbuffer could be cleared. then everything falling behind the front back plane but in front of the middle back plane would be rendered only to the depth buffer. then everything in front of the front back plane would be drawn, and that should allow for everything to be stitched together properly without artifacts. there would be two front planes corresponding to the first and second rendering groups. rendering the middle group twice only to the depth buffer teh second time allows the the depth buffer to remain coherent without polluting the colour buffer were intermediate geometry interpenetrates.

edit: took me three tries to get that description right.

it would be some decent work to impliment... especially as an optional run-time feature versus per pixel depth solving. but i will probably eventually give it serious thought. especially if the per pixel depth solving causes any sort of bottleneck.

edit: oh yeah, forgot about the back plane being hardware clipped. that makes things a lot easier. i looked at the numbers and a 100 near plane makes it without artifacts. so i really only need two back planes, or one per slice. i can actually easilly impliment this the way the system is set up simply by registering two different frustum nodes tied to the same monitor. i should be able to impliment this in minutes tomorrow, and it should run quite effectively and is much more robust than using per pixel depth solving. the only problem i can think of for using depth slicing planes is fog is outlawed i figure. anything else?

oh and... thanks a million knack!!! good idea. you are forgiven for all your nastiness. well maybe not forgiven, but you've proved to be more useful than not.

sincerely,

michael

MZ
03-26-2005, 05:41 AM
I'm afraid this thread is now unreadable without 21" monitor...

Aeluned
03-26-2005, 08:27 AM
Yes you can!
really? I was writing a shader a few months ago (maybe 6) in GLSL and I swear I couldn't adjust the fragment depth.

you can write to gl_FragDepth?

alright, my bad...

michagl
03-26-2005, 10:06 AM
Originally posted by MZ:
I'm afraid this thread is now unreadable without 21" monitor...i agree, my primary internet terminal is an 800x600 'sub-notebook' portable. i wonders if the developers of this bbs use their own system?

i'm planning to set this up as soon as possible. i thought i could just hack it together today for the short term. but system isn't frustum centric, and multiple manifolds coexist with their own local frustums which must be synchronized with a system level camera node. so i'm having to add the concept of sections at the camera node level, which is pretty deep. and since i'm going this far i'm just going to do it all right so that the frustum culling loop/recursion can manage multiple sectin bits. i will update this thread with results as soon as i get them. i have a feeling it will work out really good, unless there are minor precision problems where the opengl frustum clipper isn't pixel perfect across passes. i may just have to fudge that and hope the artifacts are minimal.

if anyone understands the numbers really well, i wouldn't mind some discussion about synchronizing fog across multiple z buffer planes if it is possible. in my immediate case i think i should be able t get by with just two slices, maybe three if i really up the scale and can handle the geometry precision. but i figure that the fogging can just start in the last slice as any closer slices will probably be too close for fogging on a clear day. on a foggy day the frustum would just be foreshortened to the point where the fog is saturated which could probably do with a single slice. so fudging for a promotional demo shouldn't be an issue.

but i'm still very interested in persuing here techniques for synchronizing the fogger states so that the fog will appear continuous even where the depth buffer isn't.

this particular system is shaping up very well... this is my first serious attempt at a large scale simulation environment. i'm generally accustumed to very local simulations and non physical systems and interface programming. i've optimized teh system pretty well. the only remaining visible cpu bottleneck is in the mesh generation step. i'm working to optimize that with SSE assembly coding which it is very well suited for. its a bit tricky because it is the work horse of the system and needs to come in many different flavors.

the only thing keeping it from a promotional demo right now is a lack of a quality data base. if anyone fancies themself a map artist and has some free time, i would be interested in sharing the details of this work and see if we can't work out a mutually lucrative arrangement. i have a very high resolution earth topology data set, but earth colour maps are much more low resolution, and various sorts of detail maps completely non-existant. the scale of earth is also too large for 64bit floating point precision. so i'm working with a smaller goal in mind. i have an artist passionate about the target world i'm attempting to realize, but he is always too busy to be of much use. of all the systems i have under my wing that i'm able to share publicly, i believe this process is a prime candidate for drawing serious interest.

Aeluned
03-26-2005, 10:33 AM
micha,

Do you have any screenshots of your system at work?

You've been talking about this system quite a bit and I'm curious to see what it's capable of, if you don't mind :)

...I like screenshots...they make me happy.

Adrian
03-26-2005, 12:24 PM
Originally posted by michagl:
i have a very high resolution earth topology data setWhat's that, the 90m SRTM dataset? I found those datasets to have quite a few data holes but maybe they've fixed those now.


earth colour maps are much more low resolution, and various sorts of detail maps completely non-existantCan you not generate your own colour maps from the height/slope data. Thats what I've done. Or are you are looking for more realistic detail than that?

michagl
03-27-2005, 08:15 AM
Originally posted by Aeluned:
micha,

Do you have any screenshots of your system at work?

You've been talking about this system quite a bit and I'm curious to see what it's capable of, if you don't mind :)

...I like screenshots...they make me happy.yeah i will put some here as son as i can get around to it.

i have some older vertex lit screenshots that still look nice of a pseudo photo-realistic shot of the himalayas.

i have some newer superior images, but they are more limited by the available data set and are not lit presently because of the nature of the world is light comes from the middle and the world is inverted, so shadows are a rarer phenomenon in such a world under natural lighting conditions. plus the data is just mock stuff i cobbled together, but i will produce some screen shots.

michagl
03-27-2005, 08:31 AM
Originally posted by Adrian:

Originally posted by michagl:
i have a very high resolution earth topology data setWhat's that, the 90m SRTM dataset? I found those datasets to have quite a few data holes but maybe they've fixed those now.


earth colour maps are much more low resolution, and various sorts of detail maps completely non-existantCan you not generate your own colour maps from the height/slope data. Thats what I've done. Or are you are looking for more realistic detail than that?the largest topology data set i have is the etopo2 projects data. it is sampled at two minute arcs i believe, and i can't find any artifacts in it, topology and bathymetry.... that is earth and seas, in signed 16bit integral format. a cylindrical mapping yields a 10800x5400 pixel map which isn't power of 2.

as for colour i definately prefer a map to ramping. even if i had hires colour maps though... the colour map has to go in texture memory, and presently i can't break the map up beyond the base geometry. which for earth to get a proper cylindrical mapping for a bipyramid, means the map can be cut 4 times verticly and once horizontilly. my card doesn't support maps larger than 512x512. so until the sister system which handles arbitrary triangular multiresolution map streaming is fairly mature, the largest colour map i can handle for earth is 2048x1024.

i'm not aiming for earth right now though. i'm going for a cylindrical world that is 200km wide and 13000 kilometers in diameter. and its pretty trivial to break up a cylindrical mapping for a cylindrical world.

michagl
03-27-2005, 12:10 PM
real quickly here are some screens... i will add some words when i get a chance to later:

http://arcadia.angeltowns.com/share/genesis19-midres.jpg

http://arcadia.angeltowns.com/share/genesis17-lores.jpg

http://arcadia.angeltowns.com/share/genesis-gaea6-lores.jpg

the first two are quite old, the last i took today.

i will say something about the third later as it isn't really representative of what the system is capable of in many respects. especially in the meshing which is as bad as it gets for surface fitting... ie: unnecesarrily craggy. the first thing to do on my todo list is to reverse the meshing which i suspect should produce one of the best possible regular meshes for surface aproximation. its just a matter of offline edge flipping.

i will add more later.

michagl
04-02-2005, 07:23 AM
what does it mean i wonder when people request screens and don't follow up?

in any case here are a couple newer images.

http://arcadia.angeltowns.com/share/genesis-hyperion4-lores.jpg

http://arcadia.angeltowns.com/share/genesis-hyperion5-lores.jpg

i'm on vacation from my development machine right now, so there have not and will not be any new development screens for about another week.

in the first image i've successfully reversed the mesh producing perhaps the best possible tiling for smoothly aproximating arbitrary surfaces. it is basicly a tightly packed grid of octagons with a vertex in the middle of each, and a vertex in the middle of the diamond created in the space between the octagons(assuming lod is constant). anyone using displacement mapping might want to look into this. the properites are really amazing. much better than a right triangle fan tesselation, and infinitely better than a regular stripping. i haven't tested it directly yet, but i have a feeling its properties for vertex shading are astounding as well.

the second screen is just the beginning of a volumetric CLOD cloud shader i'm working on in my spare time. there are typical bubble type blending artifacts, but i haven't applied a view dependant volume based pixel shader yet... which i'm thinking should cancel out those artifacts well enough, as the artifacts are really just the result of not taking into account the volume of the mesh.

ZbuffeR
04-02-2005, 08:51 AM
For realistic terrain maps,
worldwind from Nasa (http://images.google.fr/images?hl=fr&lr=&client=firefox-a&rls=org.mozilla%3Afr%3Aofficial&q=worldwind+nasa&btnG=Rechercher) will give you the most detailed data freely available that I am aware of.

About tesselation, your octogon-diamond method seem a bit strange to me, what about even hexagons, with a vertex at the center ?

Your screenshots looks very promising.

Can I ask what is the application ? Some RTS on a cylindrical world ? Simulation of a giant spaceship a la Rama ?

michagl
04-02-2005, 11:13 AM
Originally posted by ZbuffeR:
For realistic terrain maps,
worldwind from Nasa (http://images.google.fr/images?hl=fr&lr=&client=firefox-a&rls=org.mozilla%3Afr%3Aofficial&q=worldwind+nasa&btnG=Rechercher) will give you the most detailed data freely available that I am aware of.

About tesselation, your octogon-diamond method seem a bit strange to me, what about even hexagons, with a vertex at the center ?

Your screenshots looks very promising.

Can I ask what is the application ? Some RTS on a cylindrical world ? Simulation of a giant spaceship a la Rama ?i will look into the maps. we will be needing detail textures and topology maps are always useful to sample from.

well the octagon diamond method is definately ideal for my system because it is naturally arrived at algorithmicly through a series of operations generated by a clod process. but equilateral hexagons might do better as you say.

thanks for the 'promising' bit.

the application is a robust out of the box vr simulation environment. the promotional demo is a la Rama, but it is actuall gaea from John Varley's most prolific works Titan, Wizard, and Demon. Tom Clancy is quoted as saying 'john varley is america's best writer'. a quote which would probably drive most of clancy's 'conservative' followers up the walls.

check them out, best american scifi ever in my experience.

michagl
04-05-2005, 03:58 PM
it turned out that i had left in some debug code that essentially disabled filtering during the displacement process.

for what its worth i updated the image with filtering enabled:

http://arcadia.angeltowns.com/share/genesis-hyperion5-lores.jpg

the cloud in the foreground looked real bad, but i was in a hurry that day to get out of the house that day, and i just assumed it was being exagerated by the blending effect.

as it turns out, with filtering the blending overlap pretty much goes away to a large degree.

l_belev
04-06-2005, 07:56 AM
btw you dont necessary need to alter the z value of the fragments - its not widely supported yet and is expensive (extra fragment instructions). it may be sufficient if you calculate in appropriate way the z and w vertex coords in the vertex shader/program. take into account that after the vertex shader z is divided by w, so you can multiply z by w to effectively cancel this. also take into account that the z values are interpolated linearily in window space (with no perspective correction) by the rasterizer, which may be a problem if your scene contains polygons that spam too big distance in the z direction, so that this linear function get too different from whatever your function in the vertes shader is.

with perspective projection the standard function that calcs the value for the depth test is effectively 1/z (z being the third coordinate), that is z/w with w being set to z and z being set to 1 by the perspective matrix, whereas the so-called w-buffer in d3d effectively uses just z - linear function (it uses w, but w was set to z by the perspective matrix). both arent good. generally the ideal function would be log(z).

michagl
04-06-2005, 10:19 AM
Originally posted by l_belev:
btw you dont necessary need to alter the z value of the fragments - its not widely supported yet and is expensive (extra fragment instructions). it may be sufficient if you calculate in appropriate way the z and w vertex coords in the vertex shader/program. take into account that after the vertex shader z is divided by w, so you can multiply z by w to effectively cancel this. also take into account that the z values are interpolated linearily in window space (with no perspective correction) by the rasterizer, which may be a problem if your scene contains polygons that spam too big distance in the z direction, so that this linear function get too different from whatever your function in the vertes shader is.

with perspective projection the standard function that calcs the value for the depth test is effectively 1/z (z being the third coordinate), that is z/w with w being set to z and z being set to 1 by the perspective matrix, whereas the so-called w-buffer in d3d effectively uses just z - linear function (it uses w, but w was set to z by the perspective matrix). both arent good. generally the ideal function would be log(z).thanks, i will take this to heart.

for now my strategy though is to divide the scene up into depth slices as described above. the only major draw back to this aproach is you must render the slices back to front, which could cost a lot depth culling gain. but generally two slices will probably do. the closest being quite small in relative terms.

also non linear fog might be difficult to synchronize. i'm planning to use custom fog for this particular project though. not sure if i will use fogcoord or just a straight distance operation. as long as fog is only desirable in the farthest depth slice normal fog is no problem.

macarter
04-06-2005, 01:31 PM
Originally posted by knackered:

michagl, you must have a broad way of rendering everything in the background and everything in the foreground separately. That's all it takes, as you need to concentrate a large range of the zbuffer in the foreground only when rendering the foreground.
Basically, setup a frustum with the near plane being at, say 100, and the far plane at 10000 for the 1st slice, draw the scene, clear the zbuffer, then set up the frustum with the near plane at 0.1 and the far plane at 100 for the next slice, draw the scene and swapbuffers.
The standard view frustum hardware clipping will 'stitch' the two slices together seemlessly.Your frustum culling code will optimise away any unnecessary work...but if you haven't got any culling method, it won't affect the visual results, only performance.We tried this solution. The frustum clipping was far from seamless. Gaps appeared at the boundary. Overlapping the near/far planes closed the gap but caused objectionable results due to double blending of transparent polygons. Does anybody have a better idea?

michagl
04-06-2005, 07:46 PM
Originally posted by macarter:
We tried this solution. The frustum clipping was far from seamless. Gaps appeared at the boundary. Overlapping the near/far planes closed the gap but caused objectionable results due to double blending of transparent polygons. Does anybody have a better idea?well that is both good and bad to hear. i assumed if knackered was offering the advice, this was a tried and true approach... i have to give knackered credit for creative thinking though.

the only idea i can come up with is to go with my original idea before i was persuaded to rely on frustum clipping. you would have to render the transparent geometry in the invisible interstitial region to the depth buffer. that should stop the blending overlap. but this approach isn't as 'beautiful' as relying on hardware frustum clipping (the accuracy of which may differ on varying cards) though. if enough people actually wanted to use this technique then effort might be made to improve the precision of near far plane frustum clipping.

i really don't know. i haven't implimented this yet. i'm basicly just taking screenshots from far enough away as to not clip closer geometry.

zed
04-06-2005, 08:52 PM
Overlapping the near/far planes closed the gap but caused objectionable results due to double blending of transparent polygonshow?, youre not drawing the object twice, once in each partition r u.

macarter
04-07-2005, 05:19 AM
Originally posted by zed:

Overlapping the near/far planes closed the gap but caused objectionable results due to double blending of transparent polygonshow?, youre not drawing the object twice, once in each partition r u.Yes we are.

From this question I imply that you are thinking the polygons should be sliced with high precision in the CPU and the near/far planes should be overlapped so as not to cause any additional clipping in the GPU.

Yes, that should work.

zed
04-07-2005, 11:12 AM
no u should only draw each mesh once, even if it straddles the boarder devision (pick the partition that its more *in* and draw it there), if youre drawing it twice with blending of course its gonna look starnge

michagl
04-07-2005, 12:12 PM
Originally posted by zed:
no u should only draw each mesh once, even if it straddles the boarder devision (pick the partition that its more *in* and draw it there), if youre drawing it twice with blending of course its gonna look starngeyou have to draw interstitial partition nodes twice, there is no getting around that. i could offer a drawn out explanation, but i figure i would just get railed on for blowing hot air.

you have to support blended geometry, so not allowing it is out of the question.

like i originally said along way back that i was afraid this wouldn't work for this reason. but i figured if the approach was being so ardently advocated it was tried and true, and near far plane frustum culling was predictable and accurate enough to rely on.

just curious if triangles are clipped against the near and far planes in world space, or are their pixels just culled via psuedo depth?

ccbrianf
04-07-2005, 01:51 PM
Originally posted by zed:
no u should only draw each mesh once, even if it straddles the boarder devision (pick the partition that its more *in* and draw it there), if youre drawing it twice with blending of course its gonna look starngeThat only works if "each mesh" doesn't span the two regions. This was the original problem.

zed
04-07-2005, 07:22 PM
i honestly really dont see the issue, chuck in another sub region if u want
eg
near far
regionA 1-10000
regionB 1000 - 100000
regionC 50000 - 10000000

the thing is only draw the mesh once, ie pick a region and draw it there

Adrian
04-07-2005, 07:45 PM
I have implemented it, rendering the mesh only once regardless of it spanning regions. It seems to work fine.

I render all tiles except the nine around the view position with the near plane set to 100 and the far to 2500.

I then clear the depth buffer and render the 9 tiles around the view position(from near to far) with a near plane of 0.02 and a far of 400. If the view rises vertically then I have to adjust the near and far plane.

I have just two regions, the tiles are 128 units in size. I am not using blendable geometry.

ccbrianf
04-08-2005, 06:49 AM
Originally posted by zed:
i honestly really dont see the issue, chuck in another sub region if u want

the thing is only draw the mesh once, ie pick a region and draw it thereWe have two problems with this approach. We have insanely large polygons that make defining regions to wholey contain them next to impossible.

Furthermore, our sort/cull pass is a background task. Thus, sorting things into regions is problematic when combined with rapid camera rotations.

Let us know if we are missing something obvious. Thanks.

ccbrianf
04-08-2005, 06:57 AM
Originally posted by Adrian:
I have implemented it, rendering the mesh only once regardless of it spanning regions. It seems to work fine.

I render all tiles except the nine around the view position with the near plane set to 100 and the far to 2500.

I then clear the depth buffer and render the 9 tiles around the view position(from near to far) with a near plane of 0.02 and a far of 400. If the view rises vertically then I have to adjust the near and far plane.Having tiled geometry makes this possible. None of your polygons actually cross tbe boundary. We do not have this luxury.


I have just two regions, the tiles are 128 units in size. I am not using blendable geometry.Without blending, this problem goes away because overlap is not noticable other than performance.

michagl
04-08-2005, 09:23 AM
yeah try it with blending please adrian and then let us know how it turns out. just turn on blending and aet the function to blend via colour.

i'm using tilable geometry, but there will inevitably be dynamic actors which will span two regions and will need to be depth tested against the static tilable geometry.

i think zed is missing the whole point about relying on the hardware near and far clipping to automaticly splice the frustum slices together.

i could add an intermediate region and and only have it write to the depth buffer and not the colour. i believe this would work. but it just isn't as elegant of an algorithm.

i might try monkeying with the homogenous coordinates first, or just doing manual depth calculations in shaders for now. most of my shaders require calculating a view normal from the camera to the pixel anyhow, so depth is free... its just a matter of mapping it to the depth buffer and accepting any hardware hits that don't stand up to specifications.

i don't suppose there is an NV40 way to get a conditional return out of a fragment shader by querying the depth buffer is there?

ccbrianf
04-08-2005, 09:40 AM
Originally posted by michagl:
yeah try it with blending please adrian and then let us know how it turns out. just turn on blending and aet the function to blend via colour.It will work fine in this tiled case. As long every polygon/triangle you draw can be put wholey into one region, Zed's idea works.

Oops, further qualification here. As long as there is only one layer of blending, or any two blended layers all fit into one region.

Again, both at issue for us.

Zed, do you agree that these things are indeed problems?


i'm using tilable geometry, but there will inevitably be dynamic actors which will span two regions and will need to be depth tested against the static tilable geometry.Unless those dynamic actors are larger than any one region, this is not a problem. Just draw them in any one region that they entirely fit into. This restriction is key to defining the regions and sorting geometry.


{QB]i think zed is missing the whole point about relying on the hardware near and far clipping to automaticly splice the frustum slices together.[/QB]No, his plan does not rely on this. He just doesn't understand the problem of having single polygons so big that they can not be segmented into regions, and having to do the segmentation in the background.

knackered
04-08-2005, 09:52 AM
I'm not sure what you're doing wrong, but I can assure you I've done this segmented drawing approach before, not to address z buffer imprecision, but to address colour buffer imprecision - volumetric fog using fragment=backface-frontfaces (reverse_blend_subtract)....had to render it in slices. If the frustums didn't match up perfectly (and the polys they transform & clip), I would have seen a right mess.
It's going back a couple of years, but it definitely worked.

ccbrianf
04-08-2005, 10:06 AM
Originally posted by knackered:
I'm not sure what you're doing wrong, but I can assure you I've done this segmented drawing approach before, not to address z buffer imprecision, but to address colour buffer imprecision - volumetric fog using fragment=backface-frontfaces (reverse_blend_subtract)....had to render it in slices. If the frustums didn't match up perfectly (and the polys they transform & clip), I would have seen a right mess.
It's going back a couple of years, but it definitely worked.Did you move the near/far clip plane, or create user defined ones? In the former case, they will not match, in the latter, they will.

michagl
04-08-2005, 10:40 AM
i guess i will have to impliment i and see for myself.

it seems like in adrian's case the blended polygon's would overlap, but if they did then so would all blended polygon meshes on the edges i guess. why is it meshes drawn without depth testing don't overlap on the edges? i've never quesioned this. is there an algorithm that doesn't render triangle edges based on their heading in screen space? tricky tricky... i've coded a software renderer for drawing to files, but i'm still working on a fast triangle algorithm. so many considerations... i plan to use sse optimization, but it probably won't really be fast until it can do its reading and writing via hardware framebuffers.

oh, the user clipping planes might do beter, because i figure they might clip geometry directly, where as i think i've read on nvidia cards the normal clipping planes only clip fragments... in the same article i read ati cards do not do fragment clipping but do all their clipping at the geometry stage.

knackered
04-08-2005, 11:19 AM
Interesting, just did it in a test program and I do see some pixel gaps when the slice-crossing poly is almost parallel with the viewing plane.
Precision in the divide by w must be an issue.
This would certainly have blown my whole fogging algo out of the water if it'd been an issue back then....or maybe I just lived with it.
Sorry for leading you down the garden path then, michagl.
Good luck finding a better way.

michagl
04-08-2005, 04:13 PM
its a good idea. maybe it might work on some cards. maybe that really deep scenes may never be doable with a reasonable number of depth bits. maybe some one should get on the hardware people to try to make this happen.

have you tried using the user clipping planes? just set the system frustum planes a little beyond your user planes, and enable your user planes where the system planes would be. this could be more accurate because i think user planes are clipped against the geometry and screen planes are clipped in 'pixel space'.

if your test app is a one file glut type app... why not post it. i would like to try it on my machine. i've never worked with glut or writing disposable one file apps.

if it works with just some cards it is still a useful option.

sincerely,

michael

Adrian
04-09-2005, 12:09 AM
I was complaining about the lack of zbuffer precision in 2001 and 4 years later we still have the same problem. What's so difficult about adding a higher precision zbuffer? We've seen many other advances in graphics card technology but not with the zbuffer.

michagl
04-09-2005, 09:17 AM
Originally posted by Adrian:
I was complaining about the lack of zbuffer precision in 2001 and 4 years later we still have the same problem. What's so difficult about adding a higher precision zbuffer? We've seen many other advances in graphics card technology but not with the zbuffer.apparently it is because the game industry is chronicly addicted to very small environments. primarilly i figured because they are pressured to produce eye candy and not expansive seamlessly navigatable environments. plus there is the issue of discussions such as roam versus static meshing. if you don't use some kind of envolved LOD deep environments can quickly get out of hand. the game industry just isn't very creative unfortunately. and they drive hardware development. and deep scenes doesn't seem to be on the radar, if only just because that means a larger environment to populate.

i would think flight simulators would've pushed for better depth support by now though.

the exponentiality of the zbuffer might render increasing the precision virtually futile. i don't pretend to understand the numbers there. but it is just a hunch. going from 16 to 24bits doesn't seem to make much of a dent given the order of magnitude gain which should be recieved from each additional bit.

personally i like the depth slicing solution. hardware just needs to support a pixel perfect near far clipping plane in geometry space. nvidia could provide an extension to move the near and or far clipping planes into the geometry pipeline. i believe ati has always done near far clipping in the geometry pipeline, or at least i read that somewhere.

so this approach might work with ati cards, or maybe older cards that clips polygons and not pixels.

i still want to see what i can do. i plan to impliment this, though its not a top priority for me right now.

if anyone can cook up and post a simple one file glut app that relies on the near far planes for splicing i would apreciate it.

i would still very much like to try itt with user clipping planes.

zed
04-09-2005, 12:39 PM
personally ive never had a problem with the zbuffer that i couldnt solve reasonably by pushing out the near plane but

i agree that the current method of depth calculation is far from ideal
eg check here
http://www.sjbaker.org/steve/omniv/love_your_z_buffer.html
24bit depth near=1.0 far=1000000.0
then 90% of the values lie in the first 10 units!
sure u need more resolution closer to the camera but this is rediculous.
what is needed is some bastard child of z-w buffer.
ive got no mathematical ability so i have no idea what, but surely the equation aint that expensive to do in hardware

michagl
04-10-2005, 11:13 AM
Originally posted by zed:
personally ive never had a problem with the zbuffer that i couldnt solve reasonably by pushing out the near plane but

i agree that the current method of depth calculation is far from ideal
eg check here
http://www.sjbaker.org/steve/omniv/love_your_z_buffer.html
24bit depth near=1.0 far=1000000.0
then 90% of the values lie in the first 10 units!
sure u need more resolution closer to the camera but this is rediculous.
what is needed is some bastard child of z-w buffer.
ive got no mathematical ability so i have no idea what, but surely the equation aint that expensive to do in hardwarethanks for the doc... i've always wanted the actual z mapping equation.

there is this bit in the document:

"Floating point or Logrithmic Z (these are pretty similar concepts actually). This also tends to even out the distribution of bits over the Z range - but without completely eliminating the improved resolution close to the eye. SGI Infinite Reality machines do this - and claim to get the equivelent *useful* Z precision using 15 bits as a conventional machine gets with 24bits."

sounds like a start on your bastard child. more bits would always be a fine addition as well.

personally i feel like this issue warrants some sort of political action.

i also have to wonder if many games go for the third person perspective over at least a first person option because you can get by with a larger near plane in third person.

typically in video games, you don't even get remotely close to the real kind of depth you can see on any clear day... much less a view from Machu Picchu.

for this latest project, ideally i would like to be able to have a near plane of about the radius of a human head and a far plane at around 600km. i'm sure i could easilly come up with needs for a much larger divide though.

sincerely,

michael

PS: can anyone give me a link to a one file glut app with a free camera interface?

Adrian
04-10-2005, 04:42 PM
Originally posted by michagl:
and a far plane at around 600km.Does anyone know what kind of maximum view distance is realistic for Earth on a clear day? I'm using 25km at the moment.

zed
04-10-2005, 08:41 PM
Does anyone know what kind of maximum view distance is realistic for Earth on a clear day? I'm using 25km at the moment.ive read its ~4km (assuming both points are at sea level, yes its bugger all)
when i was living up in napier u can see mahia peninsuhla (200-300m high) over the ocean which is about 100km away (just measured it), from hokitika u can easily see mt cook (3764m high) 150km distance.

Adrian
04-11-2005, 01:47 AM
Ok, so view distance is all about surface curvature and not haze then.

In that case, using the formula here (http://tchester.org/sgm/analysis/peaks/how_to_get_view_params.html) the max view distance from the surface is 200 miles(~300km).

michagl
04-11-2005, 09:11 AM
i'm working in a world inside a cylinder. it is beautiful for the clod system i'm building... its only weakness is zbuffer issues.

right now i have the near plane set to 100, which is working well enough for artifact free screens... but rules out interacting with the world with any sort of human scale avataar completely.

oh yeah, forgot to say... inside a cylinder the upward curving horizon works against you, unlike on earth where the horizon disappears beneath you and is occluded by the geometry.

knackered
04-11-2005, 10:47 AM
Sorry michagl, not working with a glut project, my framework is much bigger and clumsier to deploy. Technically, it doesn't belong to me either (even though I wrote it).
I'm sure a quick search will get you a basic glut app with a camera model already implemented....maybe try modifying one of humus' older apps:-
http://www.humus.ca/
or nutty's:-
http://opengl.nutty.org/opengl.html

Anyway, I have to ask: why are you using a cylindrical world? From your screenshots, it looks a lot like the game Halo, although halo used an imposter in the far distance (billboard quad with the distant scenery rendered into a texture).
Maybe you could try something like that to bring your render load down, and concentrate the zbuffer in the foreground.
It's nice to see someone doing something mad like a cylindrical planet, I must say.

michagl
04-11-2005, 12:44 PM
Originally posted by knackered:
Sorry michagl, not working with a glut project, my framework is much bigger and clumsier to deploy. Technically, it doesn't belong to me either (even though I wrote it).
I'm sure a quick search will get you a basic glut app with a camera model already implemented....maybe try modifying one of humus' older apps:-
http://www.humus.ca/
or nutty's:-
http://opengl.nutty.org/opengl.html

Anyway, I have to ask: why are you using a cylindrical world? From your screenshots, it looks a lot like the game Halo, although halo used an imposter in the far distance (billboard quad with the distant scenery rendered into a texture).
Maybe you could try something like that to bring your render load down, and concentrate the zbuffer in the foreground.
It's nice to see someone doing something mad like a cylindrical planet, I must say.yeah, my opengl framework is big and clumsy (erhum, multifaceted) as well.

i was told a game called halo basicly ripped off ringworld.... but anyhow, i'm not really trying to build a cylindrical world... that is just secondary. the cylindrical world is just a proof of concept promotional model... the system should be able to handle a *teapot* world just as easilly, and be seamlessly navigatable.

the cylindrical world has a lot of awesome properties that really show off the 'power' of the system in every way. well lets just say it is a very flattering environment for the computational constraints of the system.

the world will also include a vaulted cathedralesque ceiling a la the biomechanical type alien space ship from the 'aliens' movie... though from a distance the ceiling would be completely invisible due to atmospheric scattering. there are also more than one hundred wound cables that hold the beast together, each comprised of smaller cables like string. the smaller component cables will be the result of a candycane displacement map. some cables slant, others go straight up, they are 5 miles in diameter, and have ecosystems of their own which dwell on their surface (especially the upper surfaces of the slanting cables).

there are blimp creatures thousands of meters in diameter which will also be clod surfaces... massive single cell amoeba type organisms which act as water pumps likewise clod surfaces.

i will give your urls a look someday... in the meantime if anyone knows of a direct link to a single file easilly guttable glut app with a free camera. please post it.

knackered
04-11-2005, 10:32 PM
lazy sod. get it yourself.

michagl
04-12-2005, 06:49 AM
Originally posted by knackered:
lazy sod. get it yourself.when i get around to it...

its not as if i wouldn't share if the tables were turned.

i'm not very good at finding stuff on the internet. at least when i do try, it usually takes to long, and 80% of the time i don't find anything close to what i'm looking for.

i might look through my newest nvidia sdk when i get a chance... but i don't remember any free camera demos in there.

or wait, there is that rainbow fog demo... and i think a volumetric fog demo as well. one of those should do.

edit: oiii, those are direct3D demos, and there are no opengl free camera demos in the sdk.

knackered
04-12-2005, 08:00 AM
Have a butchers through this lot:-
http://www.opengl.org/resources/code/glut/glut_examples/contrib.zip

michagl
04-12-2005, 01:12 PM
Originally posted by knackered:
Have a butchers through this lot:-
http://www.opengl.org/resources/code/glut/glut_examples/contrib.zip these all want a slieu of DLLs... i don't keep dlls around, and i'm not into recompiling them until i can try the binaries... i suppose at least one of these has a free camera?

so i guess, where can i download all of these dlls in one place preferably.

glut.dll, glu.dll opengl.dll, maybe glh.dll, etc...

dorbie
04-12-2005, 03:02 PM
You have to draw some stuff twice where an object transitions the division between frusta.

Adrian
04-12-2005, 03:27 PM
Originally posted by dorbie:
You have to draw some stuff twice where an object transitions the division between frusta.Correct me if I'm wrong but I think that's only necessary if the object more than spans the overlap of the frustra. If you have a big enough overlap then you shouldnt need to draw stuff twice.

michagl
04-12-2005, 06:37 PM
Originally posted by Adrian:

Originally posted by dorbie:
You have to draw some stuff twice where an object transitions the division between frusta.Correct me if I'm wrong but I think that's only necessary if the object more than spans the overlap of the frustra. If you have a big enough overlap then you shouldnt need to draw stuff twice.best i can tell there are basicly at least two algorithms envolving depth slicing being tossed around here.

one is extremely robust, but might not be possible on all hardware if any hardware.

the other requires highly constrained geometry and is really only applicable perhaps to demos or non-photorealistic environments such as scientific data visualization.

for the first the far and near planes of the depth slices must overlap perfectly to the pixel.

for the latter you have more elbow room in overlapping the planes significantly, but you can't render any kind of auxillary geometry unless it is perfectly divided along partition nodes.... other wise you will inevitably get zbuffer clipping artifacts along the seam.

the first algorithm requires geometry that falls on top of near far planes to be rendered twice, the second does not.

hope this is helpful.

sincerely,

michael

knackered
04-12-2005, 11:24 PM
Originally posted by dorbie:
You have to draw some stuff twice where an object transitions the division between frusta.No, that doesn't work - as I said, I did a test program which had no scene division logic in it, so it was simply:-


glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0f, 1.0f, 500.0f, 10000.0f);
glMatrixMode(GL_MODELVIEW);

drawScene();

glClear(GL_DEPTH_BUFFER_BIT);

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0f, 1.0f, 1.0f, 500.0f);
glMatrixMode(GL_MODELVIEW);

drawScene();...so all objects are drawn twice, and unfortunately, at oblique angles you could see the odd group of pixels being untouched by rasteriser, at the frustum crossover point.

michagl
04-13-2005, 07:38 AM
[QUOTE]Originally posted by knackered:

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0f, 1.0f, 500.0f, 10000.0f);
glMatrixMode(GL_MODELVIEW);

drawScene();

glClear(GL_DEPTH_BUFFER_BIT);

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0f, 1.0f, 1.0f, 500.0f);
glMatrixMode(GL_MODELVIEW);

drawScene();

why not try it with a user clipping plane?

if you don't know the code i will look it up.

knackered
04-13-2005, 07:43 AM
User clipping planes aren't properly supported in hardware, as far as I remember. I believe you lose a texture imaging unit when you've one enabled (uses kill fragment or something).
I've used them before, so it's not a question of not knowing how to use them.
I shall give them a try next time I'm at that PC.

sparkster
04-13-2005, 09:25 AM
Originally posted by knackered:

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0f, 1.0f, 500.0f, 10000.0f);
glMatrixMode(GL_MODELVIEW);

drawScene();

glClear(GL_DEPTH_BUFFER_BIT);

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0f, 1.0f, 1.0f, 500.0f);
glMatrixMode(GL_MODELVIEW);

drawScene();...so all objects are drawn twice, and unfortunately, at oblique angles you could see the odd group of pixels being untouched by rasteriser, at the frustum crossover point.[/QB]Have you tried slightly pushing away the 500.0f in the 2nd gluPerspective call? In a similar project, I was getting thin gaps between my depth passes and my solution was to make a tiny frustrum overlap between the passes.

Maybe using 501.0f or 510.0f would fix the problem. Might cause probs with blending though.

knackered
04-13-2005, 10:30 AM
Well yes, that's the problem - you make an overlap and blending messes up. There shouldn't be a need for an overlap.

dorbie
04-13-2005, 10:34 AM
[quote]Originally posted by knackered:

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0f, 1.0f, 500.0f, 10000.0f);
glMatrixMode(GL_MODELVIEW);

drawScene();

glClear(GL_DEPTH_BUFFER_BIT);

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0f, 1.0f, 1.0f, 500.0f);
glMatrixMode(GL_MODELVIEW);

drawScene();...so all objects are drawn twice, and unfortunately, at oblique angles you could see the odd group of pixels being untouched by rasteriser, at the frustum crossover point.</font><hr /></blockquote><font size="2" face="Verdana, Arial">Yes it's required and as you say you have two draw scene calls in there, I wasn't offering what I said as a solution, rather I was describing the technique in answer to a very specific question, i.e. the *need* to draw some stuff twice has always been there with any solution.

Objects in their own frustum can be drawn once, objects that hit multiple frusta need to be drawn > once.

In a complex free moving scene it is very difficult to avoid drawing some objects > once.

Adrian that's basically what I said, the problem as described is cracking along abutting frusta near & far clips i.e. the near and far clip operations are not exactly complimentary. To work around this the poster overlapped these, but that caused a double contribution for some fragments. The concept is sound but because of imperfect clip implementation it has issues and the workaround breaks blending.

You could try to manage this situation with dynamic clip planes but it is actually non trivial if you have a non trivial scene.

dorbie
04-13-2005, 10:40 AM
Originally posted by knackered:
Well yes, that's the problem - you make an overlap and blending messes up. There shouldn't be a need for an overlap.P.S.

It seems we're in total agreement on this.

dorbie
04-13-2005, 11:02 AM
P.S. Adrian, you're into different territory with that scheme. The overlap under discussion covers cracking, it's very small (I assumed). The overlap you imply is significant and encompasses large objects, however if you consider the requirements for such a scheme you introduce the need for pretty decent sorting and potentially irresolvable problems because one frustum does not have depth testing against the contents of another. It would probably work well for some types of app but not for others. It might be a useful solution to the cracking problem for some scenes.

knackered
04-13-2005, 01:26 PM
Ok, I've found time to test it - and it works perfectly using a single user clip plane. No missing fragments at all.

GLdouble nearPlane = 0.1;
GLdouble farPlane = 10000.0;
GLdouble boundary = 500.0;
GLdouble frustOverlap = 1.0;

glEnable(GL_CLIP_PLANE0);

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0, 1.0, boundary-frustOverlap, farPlane);
glMatrixMode(GL_MODELVIEW);
glPushMatrix();
glLoadIdentity();
GLdouble clipFront[4]={0.0, 0.0, -1.0, -boundary};
glClipPlane(GL_CLIP_PLANE0, clipFront);
glPopMatrix();

drawScene();

glClear(GL_DEPTH_BUFFER_BIT);

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0, 1.0, nearPlane, boundary+frustOverlap);
glMatrixMode(GL_MODELVIEW);
glPushMatrix();
glLoadIdentity();
GLdouble clipBack[4]={0.0, 0.0, 1.0, boundary};
glClipPlane(GL_CLIP_PLANE0, clipBack);
glPopMatrix();

drawScene();

glDisable(GL_CLIP_PLANE0);

michagl
04-13-2005, 01:37 PM
thats what i've been trying to test with all my requests for glut files...

what about this business that user planes are not supported in hardware though?

the difference i believe is the user clipping planes actually clip the triangles, that is you give it a triangle and if it straddles the plane, it is cut in two.

i will probably use the technique even if hardware does not support it well.

but this is a damn good reason to give high priority to near/far clipping support in hardware.

make sure someone at nvidia reads this thread.

after someone releases a world with deep environments, gamers will probably never seddle for less. and this aproach is likely more robust for extremely deep scenes than relying entirely on the zbuffer could ever be.

knackered
04-13-2005, 01:41 PM
By the way, would anyone from, say, nvidia like to explain why the user clip planes are more accurate than the near/far frustum planes? Is it something to do with the space in which clipping is done? User clip planes being done in world space makes the difference?

sparkster
04-13-2005, 01:52 PM
Originally posted by knackered:
Ok, I've found time to test it - and it works perfectly using a single user clip plane. No missing fragments at all.
Nice job, but...

I'm trying to understand why a user-defined clip plane can clip reliably inside different frusta whereas the near and far clip planes of those frustra are unreliable.

sparkster
04-13-2005, 01:58 PM
Originally posted by knackered:
By the way, would anyone from, say, nvidia like to explain why the user clip planes are more accurate than the near/far frustum planes? Is it something to do with the space in which clipping is done? User clip planes being done in world space makes the difference?You beat me to essentially the same question.

Aren't user clip planes defined in eye space?

sparkster
04-13-2005, 02:04 PM
Originally posted by knackered:
Ok, I've found time to test it - and it works perfectly using a single user clip plane. No missing fragments at all.
Does your drawScene use any blending? I'm wondering if maybe the user clip plane didn't really work perfectly, but instead overlapped in such a way that it redrew some fragments instead of leaving cracks.

michagl
04-13-2005, 02:11 PM
can anyone offer any insight on what kind of cost to expect for rasterizing a mesh against a user clipping plane versus without (on a variety of hardware if need be)?

of course the clipping need only be enabled for meshes straddling the plane.

edit: same here... a lot of posts popped up between my last!

michagl
04-13-2005, 02:16 PM
Originally posted by knackered:
By the way, would anyone from, say, nvidia like to explain why the user clip planes are more accurate than the near/far frustum planes? Is it something to do with the space in which clipping is done? User clip planes being done in world space makes the difference?well conceptually i figure they are better because they are designed to be used for techniques like this.

traditionally though, probably very few people if any have thought of using the traditional near/far planes for anything other than avoiding z-junk and additional fragment operations.

dorbie
04-13-2005, 02:29 PM
Clip and where it happens doesn't always match what a conceptual diagram of the pipeline does. I'd guess the difference could be caused by fundamental numerical differences once you're in the unit cube of projection space. On top of this it's not even a design objective of the API.

You're really talking about preserving sub pixel numerical representation when you've changed the normalized frustum space and depending on where you do your homogeneous divide & viewport+Z normalization that could be a virtually impossible task with a floating point representation.

knackered
04-13-2005, 11:28 PM
Yes, I suppose if accuracy wasn't in the original GL spec, then vendors can optimise like mad.
sparkster, clip planes are specified in world space, and they're transformed by the inverse of the modelview (like normals)...but then I assumed they highjacked a texture unit to interpolate the plane equation for each triangle being rasterised to figure out which pixels to kill and which not (depending on which side of the plane they're on).
No, I didn't test it with blending - I'm assuming that there's no overdraw...otherwise I can't see any use for user clipping planes at all! :)

michagl
04-16-2005, 08:09 AM
i was just reading the vertex_program extension specs:

http://oss.sgi.com/projects/ogl-sample/registry/ARB/vertex_program.txt

and under the listing titled:

"Vertex programs bypass the following OpenGL functionality:"

is this item:

"- Client-defined clip planes"

so presumably user clip planes are bypassed by the programmable pipeline... as far as this is true and unavoidable this really sucks a hell of a lot of robustness out of using this proprosed algorithm with the user clip planes.

further more, i'm not even sure how you could replace the user clipping planes in the programmable pipeline. how could you not write to a fragment without presuming the blending or alpha-test states?... which really shouldn't be necesarry. is there some way to cull a fragment from within the shader?

worst of all you have to either write multiple programs for when planes are and are not enabled, or just always assume a plane and be forced to wade through the extra computations, or at least a conditional branch.

by this point being forced to modify the depth or homogenous coordinates in every shader looks almost just as good as far as robustness is concerned.

don't shader compilers support preprocessing? or do you have to do that yourself? this could aid in restoring some level of robustness to many of these depth management options.

any thoughts please?

michagl
04-16-2005, 10:35 AM
here is some more disconcerting info from the specs:



(20) How should user-defined clipping be supported in this specification?

RESOLVED: User-defined clipping is not supported in standard vertex
program mode. User-defined clipping support will be provided for
programs that use the "position invariant" option, where all vertex
transformation operations are performed by the fixed-function pipeline.

It is expected that future vertex program extensions or a future
language standard may provide more powerful user clipping functionality.

The options considered were:

(1) Not at all. Does not work for applications requiring user clipping.
User clipping could be supported through a language extension.

(2) Support only through the "position_invariant" option, where vertex
transformation is performed by the fixed-function pipeline.

(3) Support by using the fixed-function pipeline to generate eye
coordinates and perform user clipping as specified for conventional
transformation. May not work properly if the vertex transformation
doesn't match the standard "multiply by modelview and projection
matrices" model.

(4) Project existing fixed-function clip planes into clip coordinates
and perform the clip test in clip space. The clip planes would be
transformed by the inverse of the projection matrix, which will not
work if the projection matrix is singular.

(5) Provide a 4-component "user clip coordinate" result that can be
bound by a vertex program. User clipping is performed as in
unextended OpenGL, using the "user clip coordinate" in place of the
non-existant eye coordinates. This approach allows an application
to do user clipping in any coordinate system. Clipping would not be
independent of primitive tesselation as in the conventional
pipeline. Additionally, the implicit transformation of specified
clip planes by the modelview matrix may be undesirable (e.g.,
clipping in object coordinates).

(6) Provide one or more "clip plane distance" results that can be bound
by a vertex program. For conventional clipping applications, vertex
programs would compute the dot products normally computed by
fixed-function hardware. Additionally, this method would enable
additional unconventional clipping effects. Primitives would be
clipped to the portion whose interpolated clip distances are greater
than or equal to zero. This approach has the same issues as (5).
would these documents be updated if this has changed in the mean time?

michagl
04-16-2005, 10:51 AM
all i really need to be able to do a user clip for this algorithm would be some way to reject a fragment, meaning it won't be written.

probably a safe way to do it would just be to say that for alpha blending, alpha==0 is always invisible, therefore it is possible to just always keep the alpha cull at least cull on 'equal to zero'. then setting the fragment's alpha to 0 would be equivalent to clipping it against a uniform variable clipping plane.

any better ideas?

this is not very shader friendly at all, but might be the best possible option.

at the least i need a compile-time shader preprocessor. i'm using cg. i'm assuming there is a preprocessor system, and a way to set preprocessor directives via the run-time compilation api. i will have to look into this.

bChambers
04-16-2005, 01:24 PM
Isn't there a 'kill' command you can put in your fragment shader, to kill the fragment?

ccbrianf
04-19-2005, 02:20 PM
Originally posted by knackered:
By the way, would anyone from, say, nvidia like to explain why the user clip planes are more accurate than the near/far frustum planes? Is it something to do with the space in which clipping is done? User clip planes being done in world space makes the difference?Unfortunately, on ATI, they're not. We had already tried your clip plane solution with essentially the same results as the near/far plane exact match.

knackered
04-20-2005, 03:53 AM
Oh bugger.

ccbrianf
04-20-2005, 06:09 AM
Originally posted by knackered:
Oh bugger.It does seem to be dependent on your FOV angles, though. The more narrow they become, the less likely it is that there will not be objectionable artifacts. We are simulating sensor cameras on aircraft that can sometimes have 1x1 degree FOVs. A more typical 30x40 degree FOV has much fewer, if any, detectable artifacts.

Christian Schüler
04-20-2005, 02:40 PM
It's unlikely that near and far planes of different frustra are going to align pixel perfect. The location of the planes are defined implicitly. The near plane is where the eye-space z coordinate, after going through the projection matrix and homogeneous divide, is >= -1. The far plane is at +1.

For 2 different projection matricies, A and B, aligning their near and far planes pixel perfect means you're going to demand that an eye-space z coordinate transformed by A == -1 if and only if the same coordinate transformed by B results == +1, at all times.
This is impossible to do, given the nature of floating point numbers.

User clip planes, OTOH, always compare against 0. Using the same (bit exact) coordinate as input, and the same clip plane (bit exact) as comparison for both slices, all pixels will agree whether their plane distance is >= 0 or not.