PDA

View Full Version : GPU-accelerated path rendering



Mark Kilgard
07-29-2011, 11:43 AM
www.opengl.org (http://www.opengl.org),

Normally 3D rendering and high-quality, path-based 2D rendering are distinct forms of rendering that cannot easily be mixed. Moreover while the GPU excels at 3D rendering, conventional path rendering systems for PDF, Flash, SVG, HTML 5 Canvas, TrueType, etc. have depended on the CPU for 2D path rendering.

The latest NVIDIA drivers support a new OpenGL extension called NV_path_rendering that changes this. Now CUDA-capable NVIDIA GPUs can fully accelerate path rendering. Check out:

http://developer.nvidia.com/nv-path-rendering

Get a release 275.xx NVIDIA driver (or better yet, the 280.19 beta driver) and try out the demos. Get drivers from (or search for your OS):

http://www.nvidia.com/object/winxp-280.19-beta-driver.html
http://www.nvidia.com/object/winxp-280.19-beta-driver.html

NV_path_rendering is supported for Windows XP, Vista, Windows 7, Linux, FreeBSD, and Solaris in both 32-bit and 64-bit OS flavors.

NV_path_rendering allows path objects containing line segments, quadratic and cubic Bezier segments, and partial elliptical arcs to be transformed then filled or stroked entirely on the GPU. The stroking includes all the standard embellishments such as end caps, join styles, and dashing. Unlike conventional path rendering systems where shading is generally limited to solid color, linear gradients, radial gradients, and simple 2D texturing, NV_path_rendering allows arbitrary OpenGL per-fragment shading to be applied to filled or stroked path. So you can use fixed-function, Cg, GLSL, or assembly shaders.

Rendering glyphs from fonts is a first-class feature in all path rendering systems. NV_path_rendering is no different and allows you to create path objects from either standard driver-supported outline fonts (same on every platform where NV_path_rendering is supported) or use system fonts. Instanced path rendering makes it easy to draw sequences of characters, including kerned spacing. There's support for rendering text directly from Unicode UTF-8 or UTF-16 strings. This makes high-quality font rendering truly portable across operating systems; the same NV_path_rendering font rendering code for Windows just works on the non-Windows platforms.

For a good overview of what NV_path_rendering supports, see the "An Introduction to NV_path_rendering" presentation:

http://developer.download.nvidia.com/assets/gamedev/files/An_Introduction_to_NV_path_rendering.pdf

For programmers, the "Getting Started with NV_path_rendering" tutorial shows how easy it is to GPU-accelerate path rendering. See:

http://developer.download.nvidia.com/assets/gamedev/files/Getting_Started_with_NV_path_rendering.pdf

The NVprSDK includes the source code for the nvpr_whitepaper example discussed in the "Getting Started" tutorial. Get NVprSDK from:

http://developer.download.nvidia.com/assets/gamedev/files/NVprSDK.zip

There's also a nvpr_hello_world example included in the NVprSDK that really renders "Hello world!" in OpenGL with the first-class text support. The text uses a system-supported sans-serif font, includes stroked outlining for emphasis or offsetting the text from background imagery, underlining, kerned spacing, and the text is drawn the a green-to-blue vertical gradient.

NV_path_rendering mixes seamlessly with the rest of OpenGL. So you can mix path rendering with depth-tested 3D rendering in projective views. Read the "Mixing Path Rendering and 3D" tutorial to learn how to do this. This tutorial also shows how to apply arbitrary OpenGL fragment shading to paths. This isn't something you can do with conventional path rendering systems. See:

http://developer.download.nvidia.com/assets/gamedev/files/Mixing_Path_Rendering_and_3D.pdf

Making path rendering a first-class rendering mode in OpenGL means you can use OpenGL clip planes, scissoring, depth testing, stencil testing, transformations, color/texture coordinate/fog generation, stippling, fragment shading, blending, and multisampling while rendering paths. Path rendering and 3D really are on equal terms now and fully GPU-accelerated. The same matrix transformations, textures, and fragment shaders you use for 3D rendering can be used as-is with path rendering.

Definitely check out the nvpr_tiger3d and nvpr_shaders examples.

If you want to just try out the 14 examples in the NVprSDK, I recommend running the pre-compiled examples found in the NVprDEMOs distribution:

http://developer.download.nvidia.com/assets/gamedev/files/NVprDEMOs.zip

The most interesting demo is the nvpr_svg example. It shows how GPU-accelerated path rendering with NV_path_rendering compares to other path rendering systems in both performance and quality. You can compare NV_path_rendering to Cairo, Skia, Qt, Direct2D (requires Vista and Windows 7), and the OpenVG reference implementation. You'll find NV_path_rendering is generally much faster.

For non-Windows users on Linux, FreeBSD, or Solaris, you can run these same demos but you'll have to be a bit more ambitious and build the demos from the NVprSDK but you can play with the same demos.

Most CPU-based path rendering systems have to make approximations in the interest of speed. They often approximate curved path boundaries with line segments, particularly for stroking. They typically conflate opacity computations with transparency. They take short cuts filtering color ramps for gradients or image texturing. All these simplifications and approximations diminish the overall path rendering quality.

However CUDA-capable GPUs are massively parallel architectures that make direct rendering of paths without such approximations. This leads to better overall rendering quality while at much faster rendering rates.

More questions? Try the FAQ:

http://developer.download.nvidia.com/assets/gamedev/files/NV_path_rendering_FAQ.pdf

I hope this helps.

- Mark

Aleksandar
07-29-2011, 12:27 PM
Thank you very much!

This extension is great. I really appreciate the documentation provided with the extension and the promptness of the publication.

mfort
07-29-2011, 01:33 PM
It looks really good. Seems like answer to Microsoft's Direct2D.

glfreak
07-29-2011, 01:37 PM
Great!! Thank you NVIDIA for this great extension. Keep leading the GPU industry. :) Now we got text rendering supported by the hardware, one I suggested in a different way but some ppl did not really like it.

aronsatie
07-29-2011, 02:48 PM
I hope this helps.

- Mark

Are you kidding???

There are a few things I have been praying for for quite some time. This was on the top of my list.

Alfonse Reinheart
07-29-2011, 02:56 PM
So NVIDIA decided to implement OpenVG (http://www.khronos.org/openvg/) in hardware. Only instead of implementing a cross-platform, open specification, they decided to implement an NVIDIA-only proprietary platform that will never be available outside of NVIDIA's platforms and will never be a part of OpenGL itself.

This is all very nice functionality to play around with, but this feels a lot like vendor lock-in to me. This isn't something like bindless rendering, where you can easily render one way or the other for different platforms. There's no simple switch for rendering the normal way and the NV_path way. If you're using this stuff, you're not going to re-implement an alternative rendering mechanism for it. Which means if you use it, you live in NVIDIA's world now, relying explicitly on their APIs for your application to function.

And while that's fine for some people, that's far from universal. Not everyone wants to live there, and not everyone can even if they wanted to. Isn't OpenGL's greatest (perhaps only) strength the fact that it is cross-platform? That you can run just fine on Windows, Linux, MacOSX? That you can run on NVIDIA or AMD equally (bugs aside)?

So while I can understand the "this is great to have!" sentiment from a features perspective, I can't say I like the encouragement for developers to bind themselves tightly to NVIDIA hardware.

This is fine functionality and all, but why wasn't it part of an OpenVG implementation instead of something proprietary?

glfreak
07-29-2011, 03:22 PM
This is fine functionality and all, but why wasn't it part of an OpenVG implementation instead of something proprietary?

Where's OpenVG32.dll or whatever it's called under whatever platform? How can I use it? Any PC implementation?

How would it coexist/work with OpenGL window?

This extension will soon be adopted by the ARB, probably will become EXT...

In short words, it's a First Class Extension that will make OpenGL a more kick-ass API!!!

Gedolo
07-30-2011, 01:54 PM
I would like to have 3d paths please.

That the paths themselves can be in 3 dimensions.
Would be nice to be able to build scenes with them.

Also regarding OpenVG.
As api it actually wasn't very good.

Having a format independent api will allow much more flexibility.
This will be much more usable for all kinds of formats.

Also making the font stuff codepoint independent for fonts would also be a nice flexibility. I'm not saying the current stuff with UTF-8 should go but that there should be room for future encoding systems. Independent of the api.

I don't see groups of paths anywhere mentioned in the presentation. Instancing is great, please keep that.

Groups (especially if combined with instancing) would really be handy, especially when using fonts and doing text.



Reading through An_Introduction_to_NV_path_rendering.pdf

Please make a single, general and uniform circular/elliptical arc primitive.
I don't think using path format is a great idea.
We already have OpenVG.
And these formats continue to evolve. You just need to have universal usable building blocks that can be used to make everything from. Please separate the api from formats. You can always introduce formats in an SDK or software library.

About slide 17.
It would be nice to change the end caps to round or square and have a parameter that says what the length of the caps should be. And it should be possible to use another path or group as caps.

I would like to be able to specify that a path borders should end in a path or group used as caps but should not come back out.

It seems like the join styles can also be shortened to two, with a parameter that tells how far to cut the rest of the sharp tip off.

All these 'example' dash styles seem like a can of worms.
(When is it enough, should there be more added? Yes/No It never stops. There is always going to be another style.)

Being able to intelligently define a pattern and use that along a path should be able to be used to recreate all these styles.

Don't make too much different path functions that can be done with a smaller amount of more universal ways of doing things.

This is an api for making paths. It's not a data format but a bunch of functions. It needs to capture essence of and the path primitive. It needs to be format independent.
The decorations of paths need to be constructed from the primitives, not be them.

OpenGL used to have a fixed function pipeline but it didn't worked out. Please don't make the same mistake.

You can always introduce standard styles with an SDK that adds these more constructed stuff. Just as there are libraries for OpenGL.

This stuff is great but it needs to be improved.

There are a lot of generalizations possible that can reduce the amount of primitive functions while making them more flexible and powerful.

Gedolo
07-31-2011, 07:55 AM
After reading some more.
Some additional things come to mind while reading
An_Introduction_to_NV_path_rendering.pdf



At slide 12.
It would be better to have functions that allow to change the origin to a relative origin and back again instead of having a dozen of functions with relative origin.
Being able to name the origins would be nice too.



At slide 13

Instead of cramming everything in GL_NV_path_rendering

Please do an extension for the formats that includes the version in the title. The version number of a data format is important and necessary. It really is. The underscore between the two numbers in the svg and ps extension name is also important. It allows to make a difference between what is the major and what is the minor number.

Add two new extensions and keep most non-specific functions separate from file formats:

GL_NV_format_SVG_1_1_
This way everything that has to do with svg specifically can be put in here.

(The last underscore allows to make a difference between the version of the format implemented and the version of the extension.)

If there is a underscore behind it, it's a version number of the format. If a number is the last thing in an extension, it's the extension number. Being able to do this allows for changing specifications without breaking things. Supporting several file formats and versions of the same file format alongside each other without interfering with each other and extension/OpenGL api specification evolution/numbering.

GL_NV_format_PS_?_?_
This way everything that has to do with postscript specifically can be put in here.



At slide 35
I would like to see more general intelligence placing of stuff instead of just glyphs.
(Kerning <> Using bounding boxes when wanting to place a series of objects next to each other without overlapping. With relative start location and direction. Allowing objects to have a separate box with user definable origin and size, origin relative to the glyph, as a bounding box.)

It doesn't have to be text or glyphs. Just paths and other things e.g. textures or stuff from the 3d rendering pipeline.

Options in the clone dialogue of Inkscape come to mind.



slide 54
Being able to fill a box or the inside of a closed path with figures would be nice too. Something like the flowing text in Inkscape but more general.

Able to specify the relative begin and direction of flow.
e.g. origin: left, bottom
direction: right,up

englis: origin: left, top direction: right, down
chinese: origin: left, top direction: down, right

This would start in the left bottom corner. It would place the first thing, then place the next thing right of the first thing when the 'row' is full. It will go one 'row' up and start from there.

Lots of stuff done with fonts also has a general equivalent.
I would like to be able to use that instead of this specific subset where the restriction (that the path is a glyph) has almost no difference to any other path or group of paths.



Maybe it would be nice to move all font stuff to a separate extension? Which then could contain all the stuff about fonts.
Something like:
GL_NV_path_rendering_font

These modularities can make it much easier to develop the different part of the specification. (And getting the specifications promoted or even in core if wanted.)

Gedolo
07-31-2011, 08:20 AM
Another discussion about this:

Path rendering - OpenGL.org Discussion and Help Forums (http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&amp;Number=300881#Post3008 81)

YarUnderoaker
08-01-2011, 02:52 AM
I'm trying to port nvpr_font_file demo to Pascal. All works fine, except

glPathParameteriNV(pathTemplate, GL_PATH_STROKE_WIDTH_NV, 50);
glPathParameteriNV(pathTemplate, GL_PATH_JOIN_STYLE_NV, GL_ROUND_NV);
both command cause GL_INVALID_OPERATION.
In this time I can't check, but may be is this due to lack of library freetype6.dll in project folder? How it interact with application?

ScottManDeath
08-01-2011, 12:02 PM
It seems that the extension ignores geometry and vertex shaders, so now I'm wondering how one would do e.g. (geometry shader) instancing to replicate a path into multiple layers of a 2D texture array without having to render it N times and bind each individual layer to make it work and changing e.g. the modelview matrix for each pass. With triangles one would use GS instancing ...

However, using gl_ViewportIndex for the ARB_viewport_array extension requires a GS in order to route stuff into the different viewports, which can be handy when e.g. rendering into 2x2 subdivided viewport. So how would I be able to render paths into each viewport without having to render them 4 times?

Gedolo
08-01-2011, 01:01 PM
Fonts are (in a more limited way) already possible.
http://code.google.com/p/freetype-gl/

The little library that uses shaders and other basic building blocks does speak to me as a good solution. Doing this for paths would not create problems if licence of the font library is free enough.

Also I don't understand why paths aren't geometry?

Aren't vertices also kinda like paths?

I see why not using the tesselation pipeline but not why the path stuff couldn't be integrated into geometry shader.
That will probably ask a new generation of hardware.

YarUnderoaker
08-01-2011, 01:34 PM
both command cause GL_INVALID_OPERATION.
I added GL.PathCommandsNV(pathTemplate, 0, nil, 0, GL_FLOAT, nil) before and error is disappeared.
I guess, in SDK demo this line must be too, because after I added it, stroke width start look better.

Mark Kilgard
08-01-2011, 07:31 PM
Also I don't understand why paths aren't geometry?Very good question. Paths are a well-developed concept in resolution-independent 2D graphics. PostScript, PDF, SVG, Flash, TrueType, Quartz 2D, Cairo, Skia, XPS, etc. all have the concept. It helps to understand what a path is.

If you are super-familiar with OpenGL, but haven't really been exposed to path rendering, it's really easy to think "everything is just triangles, right?" and really miss what makes path rendering standards distinct. Start by divorcing the notion of a path from concept of a geometry batch (commonly, a bunch of triangles, often in a mesh, as in standard OpenGL).

Paths really are a sequence of path commands. Examples of path commands are MoveTo, LineTo, QuadraticBezierCurveTo, ArcTo, ClosePath, etc. The palette of available commands varies by path rendering standard, but they all have some command commands. All at least have MoveTo, LineTo, and QuadraticBezierCurveTo.

There are plenty of 2D authoring tools that let you author path content. Adobe Illustrator and Inkscape are the two best known.

A path consists of zero or more trajectories. A trajectory is nothing more than a connected sequence of 2D segments (potentially curved) that are connected to each other. You might also call this a spline. Each trajectory or spline can be closed (meaning the start point of the trajectory is the same as the end point) or open (meaning the start and end points are distinct). A closed trajectory or spline is often called a contour. Math types might call a contour a 1-dimensional manifold.

Once you have this basic notion of a path, you are ready to render a path. That can be done by either filling or stroking the path after its has been transformed from path space to window/device space.

Filling a path is like the idea of "coloring between the lines" in a coloring book. When a path is not closed, it is "forced" to be closed by a line segment connecting the start and end point. But that analogy only goes so far because paths can self-intersect each other and a single path might contain multiple contours. As you'll see, it also potentially matters whether the path winds clockwise or counter-clockwise. This makes it necessary to determine in a rigorous way whether a point in space is "inside" or "outside" the fill of a path. You do this by computing the winding number of a point in the 2D plane with respect to the path. You count how many times the path winds around the point. Another way of doing this is casting a ray in an arbitrary direction to infinity and counting the net number of times a ray cross the path where counter-clockwise crossings are counted as positive (+1) crossings and clockwise crossings are counted as negative (-1) crossings. (These two interpretations are identical mathematically.) If the number of crossings is non-zero for a point with respect to a path, the point is within the path. Otherwise, the point is outside. Alternative rules exist as well. The other common rule is the even-odd rule where you are inside if the winding number of a point w.r.t. the path is even (and outside if odd). Other rules might ask if the winding number modulo some power of 2 is non-zero; in this sense, the even-odd rule is simply a "modulo 2" rule.

Notice that filling is a "global" operation. You can't just look at a subsequence of path commands in a path and be able to determine whether a point is inside the filled region of a path or not. Because some sequence you are not considering could "cancel" or "add" the point back into the filled region of the path. You've got to consider the whole path to make an accurate determination of what's inside and outside the path.

Stroking a path is quite different from filling a path. Instead of computing the winding number of a path, the analogy is taking a pen (like a calligraphy pen) and pulling it over the path (raising it off the paper when MoveTo commands are encountered and lowering the pen at the new location) such that the pen's tip is always orthogonal on the path and centered on the path. Essentially you are "stroking" out a region of 2D path that is within the stroke of the path.

There are a lot of (important) embellishments to stroking involving stroke width, end caps, join styles, dash patterns, and dash caps that complicate what it means to be "inside" the stroke of a path. While I'll only mention these stroking embellishments in passing, they are important to most major path rendering standards. For this reason, NV_path_rendering fully supports them.

Notice there's no concept of points/lines/polygons (the familiar geometric primitives in 3D rendering). Indeed, the boundaries of paths are actually defined by (parametric sub-regions) of polynomial curves (rational polynomial curves in the case of partial elliptical arcs). This is *way* different from 3D graphics where every primitives edge is "just" a line segment.

The implications of the higher-order nature (in the polynomial sense) of stroked or filled path boundaries are substantial. With filling, two path commands, each specifying a cubic Bezier segment, might intersect. That's the intersection of two 3rd order curves. Just determining where these two curves might intersect would require solving a 6th order (3+3) equation. In the case of a stroked path, the boundary of a stroked path of a cubic Bezier segment is specified by a 10th(!) order curve.

I hope this helps explain why paths aren't simply "geometry" in the simple sense supported by 3D APIs.

None of what I'm saying is unique to NV_path_rendering. All path rendering standards deal with these issues and practicalities. This complexity is one of the reasons GPUs have not accelerated path rendering particularly effectively. NV_path_rendering seeks to change that.


Aren't vertices also kinda like paths?Another good question, but, no, vertices are really NOT kinda like paths. Paths are typically defined by control points associated with the path's command sequence. Each control point is defined by two path coordinates. Each control point may specify an absolute 2D position (x,y) or a relative 2D position (dx,dy) that is relative to the end point of the prior path command in the path's command sequence. Some path commands use path coordinates for purposes other than specifying 2D control points. Commands to specify partial elliptical arcs specify flags, radii, or angles as path coordinates. The SVG partial elliptical arc command is an example of such a path command.

Even if you discount the non-control point path coordinates used for partial elliptical arc parameterizations, you'd be naive to confuse a vertex with a control point. Whereas vertexes are boundary points on a geometric primitive, a control point might not actually even be on the path. For example, the quadratic Bezier segment command in path rendering specifies two control points. The first is an extrapolating control point NOT part of the path segment's curve.

A path is really a convenient way for artists to specify a region within 2D planar space. Paths are NOT just a mesh of triangles. Paths are resolution-independent. Paths have higher-order boundary representations. Paths can be self-intersecting and have multiple disjoint pieces and holes. A path can be unbounded in the number of path commands (and corresponding path coordinates) that make up a path.

As great as vertex shaders and geometry shaders are for 3D geometry specified as batches of points/lines/triangles, paths just aren't that sort of animal.


I see why not using the tessellation pipeline but not why the path stuff couldn't be integrated into geometry shader.Geometry shaders operate on geometric primitives constructed from a fixed number of vertexes and operate by emitting vertexes that are likewise constructed into similar geometric primitives.

Paths aren't triangles and don't have vertexes so paths can't be feed to a geometry shader. Likewise for vertex shaders.

Perhaps it is worth saying that NV_path_rendering (as with pretty much all path rendering implementations) do NOT operate by breaking paths into triangles and then rasterizing the resulting triangles. Doing so isn't efficient because tessellation is a fragile (in the sense that it must be done very carefully in terms of numerical operations), sequential, and ill-suited for the curved and resolution-independent boundaries of paths. Most path rendering implementations operate through scan-line rasterization. NV_path_rendering is a bit different because it harnesses the massively parallel and pipelined nature of CUDA-capable GPUs to directly evaluate the stroking and filling determinations for transformed paths.

- Mark

Mark Kilgard
08-01-2011, 07:59 PM
both command cause GL_INVALID_OPERATION.
I added GL.PathCommandsNV(pathTemplate, 0, nil, 0, GL_FLOAT, nil) before and error is disappeared.
I guess, in SDK demo this line must be too, because after I added it, stroke width start look better.

Thanks for the bug report. Yep, the pathTemplate object wasn't being created first so you couldn't set parameters on it. The fix is exactly what you describe.

Here's a suggested C code fix, similar to what you found worked:



/* Use the path object at the end of the range as a template. */
pathTemplate = glyphBase+numChars;
glPathCommandsNV(pathTemplate, 0, NULL, 0, GL_FLOAT, NULL);


Yes, with this fixed, you actually get to see the stroking.

Expect this to be fixed with the next NVprSDK.zip update.

Thanks again. Otherwise, have you found the simpler NVprSDK demos port to Pascal pretty cleanly? Probably nvpr_svg is too much work because of the extensive C++, but many of the others should be simple enough to translate.

- Mark

YarUnderoaker
08-02-2011, 07:05 AM
have you found the simpler NVprSDK demos port to Pascal pretty cleanly?
Yes, everything is clear and simple. I have low skill of using Visual Studio, but I could easily compile examples.
Thank you very much for this great extension.

elFarto
08-11-2011, 04:25 AM
Hi Mark

Thank you for this extension.

Have any of the other Khronos members expressed interest in moving this extension to EXT or ARB status? Could this extension be implemented on (for example) AMD's hardware, or does it require some specific piece of functionality only found on NVIDIA's GPU?

Thanks & Regards
elFarto

Mark Kilgard
08-25-2011, 03:32 AM
elFarto,

> does it require some specific piece of functionality only found on NVIDIA's GPU?

Yes, NV_path_rendering requires your GPU to be CUDA-capable.

- Mark

Gedolo
03-04-2012, 05:14 AM
Make it so it does NOT require compatibility.
Seriously, make sure it only requires core!!!

kyle_
03-04-2012, 08:49 AM
Well, CUDA-capable is pretty vague. Other vendors could probably go ahead and implement it (technically). Its not like nvidia hardware is made from rainbows and sunshine dust.
Then again, i don't really feel GL is good place to cram such features, id rather it was put elsewhere.

It was already shown before that such stuff is doable (i think Hoppe from ms did some research on svg rendering on GPU some time ago) and is certainly pretty need thing to have (Inkscape with this in backend would be pretty cool).