Hello, i was taking a coffee this noon, and as always i started to think about what i could add to my engine… i’ve been thinking about how to use texture fetching in vertex programs, and well suddenly i realized that perhaps displacement mapping in a vp is not a good idea (yet)… well it is, very funny and such, but you couldn’t interact easily with meshes with displacement map.
As all displacement is made in the GPU you don’t have access to actual topology of geometry and, for example, colision detection is impossible. When superbuffers arrive and we can retrieve vertex positions easily (fast) from gpu then we will be able to use that displacement maps for something else that just nice visual effects
Of course you can do approximations (as knowing the bounding box of the displaced mesh and such but u still don’t have an exact triangle-triangle colision).
Well, it was just a thought, what do u think?
IMHO it depends on what you want to do with the displacement mapping.
Let’s say you want to use it to create dynamic heightmaps for terrains. In that case you could add an additional pass in your program that renders a depthmap from an orthographic view from the topand looking down. You could then use that depthtexture for collision detection. This is potentially faster then doing collision detection on the CPU, depending on the number of triangles.
This extra pass is not necessarely that expensive. I read that the Geforce 6 has a double-speed Z-only render functionality.
Vertex texture fetch also has applications in general processing (see gpgpu presentations). But that’s not necessarily very applicable to 3d engines .
Also you can think about it like this… If you have a 10,000 poly character model would you transform every vertex and then collide all those with your environment, or worse, other objects? Not only would it be expensive as it is, it would probably produce an insane number of contact points for physics calculations. Instead most people use a much lower poly model which has less bone influence, but still approximates the shape of the model (fewer calculations etc). The same can be said of shadow rendering, though less often probably.
It is true though that displacement mapping isn’t half as awesome without adaptively tesselated surfaces (even just planar tesselation…). I just recently looked at nvidia’s adaptive stuff and it looks pretty awesome . I hope they bring it back en force at some point.
Most games and virtual environments use collision proxies for their actors, rather than full mesh collision. If collision was per-triangle, you couldn’t even do mesh skinning in a vertex shader (which has been done for a long time).
Most of the time, when people talk about using displacement mapping, they’re wanting to use it for adding details. Like bump mapping, but with actual geometry.
The problem with the height field approach is that it doesn’t save anything. It would only save memory if the displacement map was relatively low-resolution, compared to the overlying geometry. And, if it’s low resolution, why is the geometry so high-res? You may as well be passing low-res geometry with the full data.
If the displacement map is high-res, then you’re transfering upwards of one pixel’s worth of height data per vertex. At which point, the height may as well be a vertex attribute, right? Texture access in a vertex program isn’t nearly as efficient as using a vertex attribute directly.
Now, you could procedurally generate the heights based on some seed data. But, then again, you aren’t using a displacement texture (unless it is your seed data), so you don’t need per-vertex textures at all.
The nice thing with height fields is that you can add/subtract/scale them in a very consistent manner.
An alternative is to build a “height map” that is very narrow but tall, with one sample per vertex. That way, you can apply a morph target through texturing in the vertex shader. You can put more morph targets going across the u axis, or you can put time going across the u axis, and get vertex animation “for free”.
There are lots of other cool things you can do, like sending in skeletal poses as position and quaternion (or matrix) using floating point textures, and blend/animate THOSE, all in the shader.
There are lots of other cool things you can do, like sending in skeletal poses as position and quaternion (or matrix) using floating point textures, and blend/animate THOSE, all in the shader.
nVidia didn’t recommend this, due to the slowness of vertex texture accesses.
i really don’t know much about displacement mapping in realtime, so maybe i’m wrong, but couldn’t you do a procedural displacement map (like waves in water, for example) and then simulate that in your physics with the same equation?
nVidia didn’t recommend this, due to the slowness of vertex texture accesses.
Yes, and they said pixel shader 2.0 was too slow to use on the first GeForce FX cards, too. That doesn’t mean ps 2.0 is a bad idea.
It’ll be faster on later hardware. Having an entire character animation system in hardware has significant value. Once you have the structure, you can pull back pieces to the CPU if you want it, but as the hardware comes down the pipe, increased parallelism is available.
Another limitation is that you can only use four separate textures from the vertex stage, although if they’re 3D and you can control filtering, you could get around this with some hacks.
I also find that if I design for hardware acceleration first, and then do the software fallback, the structure of the resulting code is often better than if I go software first, and then try to push it to hardware.
That’s still a lot of texture accesses inside a dynamic loop per vertex. It might be worth it, except with using quaternions you already can fit a TON of bones in to all the constant registers (which can be indexed…).
Unless you were animating something pretty detailed which had a ton of different bone influences, or have something that you need the constant registers for that would be even slower, constant registers would probably be the better solution.
As for physics, yes you can do much more detailed stuff in the vertex shader, but most of the application is still probably going to be in the pixel shader, which was available before.
Yes, and they said pixel shader 2.0 was too slow to use on the first GeForce FX cards, too. That doesn’t mean ps 2.0 is a bad idea.
I’m not saying it’s a bad idea; I, for one, would dearly love to get my matrices out of uniforms and into a memory buffer. I’m saying that it is a bad idea today. As Adruab pointed out, it is still better to use uniforms. Until texture accesses start approaching reasonable speed at the vertex level, it will continue to be better to use uniforms.
How many uniforms does the 6800-level hardware offer, anyway?
The point is not that I can fit the entire skeleton in uniforms. I can.
The point is that I can fit an ENTIRE ANIMATION (all the key frames, not just one) into a texture. All poses, for all samples. Leaving the CPU to specify just an index into the animation along the time scale.
Fit a bunch of animations into textures, and use blending, to blend animations – still all on the GPU.
Most exciting, to me: this way, I can actually do vertex animation (morphing) on the GPU, too. Not all animations suit themselves well to current skinning implementations. Things like a rotund belly jiggling, or folds of cloth flapping, or fix-ups and bulges around elbows, arm pits, and the like are much better done with vertex animation, and can now be seamlessly applied by the GPU; no CPU involved.
Fit a bunch of animations into textures, and use blending, to blend animations – still all on the GPU.
That’s a good way to kill your GPU and get yourself vertex T&L bound.
Maybe your animation system isn’t too complicated, but some animation systems blend between multiple animations, and can blend with fairly arbitrary poses (some of them possibly generated by physics, ie: ragdoll).
I agree that I’ll probably go T&L bound on current hardware. That’s fine. My point is that I don’t see CPUs gaining a massive lot of performance – we’re been at an almost stand-still for two years now! Meanwhile, I see GPUs scaling up a fair bit into the future.
To run networked distributed rigid body dynamics simulations and advanced character AI on the CPU, I’d want the 25% of the CPU spent on character animation to go “elsewhere”. That “elsewhere” is likely the GPU. I don’t want that “elsewhere” to be a second CPU, because it would still fight for RAM bandwidth with the other algorithms.
You spend 25% of your CPU on character animation? Maybe you should tell your artists that vertex animation is not a viable technique and use bone animation like everybody else.
Ah, yes I missed the fitting animation thing… It’s interesting, but doesn’t seem very necessary to me. Just relying on the linear interpolation seems a bit limiting to me. It removes the possibility for advanced forms of animation driving (e.g. procedural controls to remove feet penetrating the ground and the like ). Plus since the blending is invariant for the entire model, so that’s a lot of redundant calculation . For morph targets that could be useful… any specific applications that couldn’t be done by using bones or some other influence based method more efficiently?
Morph targets can be done as additional per-vertex data (blend between these 4 vertices to get the answer, etc). The only limitations are in the number of per-vertex parameters, but that’s still pretty large (16).
