PDA

View Full Version : OpenGL Registry specs inconsistency and bugs



Night Nord
02-04-2012, 09:20 AM
I'm writing my own opengl extensions wrapper for private use.

Initially I've tried to parse system's GL/gl{x,}{,ext}.h files to obtain information about extensions, as I believed that it would be much easier to parse code with known rules that specs from opengl.org without public syntax. However, after some work, I realized that too much information is simply missing from headers files and I was forced to hack upon spec "format".

Well. I really don't believe that specs should be in XML, they should be in RDB (database) instead, as keeping consistency with so much intersecting data in text files is, obviously, not an easy task, while database restrictions will keep this consistency for you. Anyway, I'm writing here to post a few discovered inconsistencies in spec files, that may or may not be bugs and a few obvious bugs.

As specs are essential for any opengl support on non C/C++ languages (opengl.org doesn't provide files for other languages), they should be consistent and relatively easy to parse. At least, they should follow same conventions from one part to another.

There is my changes to specs that I've made to make my parser work (in fake diff format):


enumext.spec/ARB_gpu_shader5:
- use ARB_texture_multisample MAX_VERTEX_STREAMS
+ use ARB_transform_feedback3 MAX_VERTEX_STREAMS

enumext.spec/VERSION_4_0:
- passthru: /* Reuse tokens from ARB_transform_feedback2 */
- use ARB_tessellation_shader TRANSFORM_FEEDBACK
- use ARB_tessellation_shader TRANSFORM_FEEDBACK_BUFFER_PAUSED
- use ARB_tessellation_shader TRANSFORM_FEEDBACK_BUFFER_ACTIVE
- use ARB_tessellation_shader TRANSFORM_FEEDBACK_BINDING
- passthru: /* Reuse tokens from ARB_transform_feedback3 */
- use ARB_tessellation_shader MAX_TRANSFORM_FEEDBACK_BUFFERS
- use ARB_tessellation_shader MAX_VERTEX_STREAMS

+ passthru: /* Reuse tokens from ARB_transform_feedback2 */
+ use ARB_transform_feedback2 TRANSFORM_FEEDBACK
+ use ARB_transform_feedback2 TRANSFORM_FEEDBACK_BUFFER_PAUSED
+ use ARB_transform_feedback2 TRANSFORM_FEEDBACK_BUFFER_ACTIVE
+ use ARB_transform_feedback2 TRANSFORM_FEEDBACK_BINDING
+ passthru: /* Reuse tokens from ARB_transform_feedback3 */
+ use ARB_transform_feedback3 MAX_TRANSFORM_FEEDBACK_BUFFERS
+ use ARB_transform_feedback3 MAX_VERTEX_STREAMS

reason:
There is no MAX_VERTEX_STREAMS in ARB_texture_multisample
(and enum.spec -> enumext.spec converter seems to ignore extensions with digits)




enumext.spec/NV_fog_distance:
- use TextureGenParameter EYE_PLANE
+ use VERSION_1_1_DEPRECATED EYE_PLANE

enumext.spec/NV_register_combiners:
- use BlendingFactorDest ZERO
- use DrawBufferMode NONE
- use GetPName FOG
+ use VERSION_1_1 ZERO
+ use VERSION_1_1 NONE
+ use VERSION_1_1_DEPRECATED FOG

reason:
There is no such extensions, this is a categories in original
enum.spec (which is one big inconsistency at the first place.
No doubt there is a problem to properly convert it).




enumext.spec/ARB_tessellation_shader:
- use VERSION_1_1 QUADS
+ use VERSION_1_1_DEPRECATED QUADS

enumext.spec/VERSION_3_0:
- use ARB_framebuffer_object INDEX
+ use ARB_framebuffer_object_DEPRECATED INDEX

enumext.spec/VERSION_3_0_DEPRECATED:
- use ARB_framebuffer_object TEXTURE_LUMINANCE_TYPE
- use ARB_framebuffer_object TEXTURE_INTENSITY_TYPE
+ use ARB_framebuffer_object_DEPRECATED TEXTURE_LUMINANCE_TYPE
+ use ARB_framebuffer_object_DEPRECATED TEXTURE_INTENSITY_TYPE

reason:
Let's mark deprecated stuff as DEPRECATED explicitly. While
it's possible to look for _DEPRECATED suffix while resolving
links, specs are already hard to parse properly enough.




enumext.spec/SGIX_polynomial_ffd:
- FfdMaskSGIX enum:
- TEXTURE_DEFORMATION_BIT_SGIX = 0x00000001
- GEOMETRY_DEFORMATION_BIT_SGIX = 0x00000002
-
- SGIX_polynomial_ffd enum:
- GEOMETRY_DEFORMATION_SGIX = 0x8194
+ SGIX_polynomial_ffd enum:
+ TEXTURE_DEFORMATION_BIT_SGIX = 0x00000001
+ GEOMETRY_DEFORMATION_BIT_SGIX = 0x00000002
+ GEOMETRY_DEFORMATION_SGIX = 0x8194

reason:
There is no such extension in OpenGL Registry, and it seems
to be another bug of enum.spec->enumext.spec converter




enumext.spec/SGIX_ycrcb_subsample:
- PACK_SUBSAMPLE_RATE_SGIX = 0x85A0
- UNPACK_SUBSAMPLE_RATE_SGIX = 0x85A1
- PIXEL_SUBSAMPLE_4444_SGIX = 0x85A2
- PIXEL_SUBSAMPLE_2424_SGIX = 0x85A3
- PIXEL_SUBSAMPLE_4242_SGIX = 0x85A4
+ use SGIX_subsample PACK_SUBSAMPLE_RATE_SGIX
+ use SGIX_subsample UNPACK_SUBSAMPLE_RATE_SGIX
+ use SGIX_subsample PIXEL_SUBSAMPLE_4444_SGIX
+ use SGIX_subsample PIXEL_SUBSAMPLE_2424_SGIX
+ use SGIX_subsample PIXEL_SUBSAMPLE_4242_SGIX

reason:
Conflicting enums, later extension should reuse enums
from prior one.




glx.spec/SGIX_hyperpipe:
- passthru: typedef struct {
- passthru: char pipeName[GLX_HYPERPIPE_PIPE_NAME_LENGTH_SGIX];
- passthru: int channel;
- passthru: unsigned int
- passthru: participationType;
- passthru: int timeSlice;
- passthru: } GLXHyperpipeConfigSGIX;
- passthru:

+ passthru: typedef struct {
+ passthru: char pipeName[GLX_HYPERPIPE_PIPE_NAME_LENGTH_SGIX];
+ passthru: int channel;
+ passthru: unsigned int participationType;
+ passthru: int timeSlice;
+ passthru: } GLXHyperpipeConfigSGIX;
+ passthru:

reason:
Despite being unformatted passthru, it's even not consistent
with previous usages. That's not funny - situation is bad
enough already.




enumext.spec/VERSION_4_0:
- use ARB_gpu_shader5 MAX_VERTEX_STREAMS
+ use ARB_transform_feedback3 MAX_VERTEX_STREAMS

reason:
Let's import from some one place


Maybe it would be better to post it on bugzilla?

And a few thoughts about specs themselfs.
1. Passthru. Is. Evil. Really. There is another languages, not only C/C++, you know - and you are hiding important information (like typedefs or #ifdef protections against external headers) within non-formatted passthru. All vital information should be marked explicitly. Types, probably, should be in another spec.

2. Enumext.spec is ext-grouped, but function-specs (gl.spec, glxext.spec) are ext-grouped only partially (via bad-named tags newcategory:, endcategory: (later only in glxext.spec))

3. Typedefs (in passthru's) are protected by #ifndef's only partially, SGIX_hyperpipe struct's are non-protected and protected magically by internal script logics (probably by using newcategory:, endcategory:).

4. enum.spec is... well... strange. Not sure that it's usable outside Khronos Group in any way - there is groups-by-ext and groups-by-logic.

5. Having significantly different syntax for function-specs and enum-specs is really frustrating.

6. If you really like text-files so much, IMO it would be better to use YAML instead of XML for future use, as it's much more readable by humans (and easier to parse).

7. There is enum "use"'s and "alias"'es (sharing same enum value by different names). Some aliases are resolved during enum.spec->enumext.spec conversion, but others are left as-is. That's strange.

BTW, I've managed to import almost every bit of information from specs files into internal perl structures, so it should be relatively straightforward to push this information into other format, XML, YAML or RDB...

Alfonse Reinheart
02-04-2012, 09:48 AM
Maybe it would be better to post it on bugzilla?

Lol. That would assume that someone actually read the Bugzilla. They don't mark bugs fixed, or even that they've read them (changing the status from NEW to something else).


BTW, I've managed to import almost every bit of information from specs files into internal perl structures, so it should be relatively straightforward to push this information into other format, XML, YAML or RDB...

Been there, done that, have a public repository (https://bitbucket.org/alfonse/gl-xml-specs) for it. Oh, and I have an XML version of it too ;)

Night Nord
02-04-2012, 10:48 AM
Lol. That would assume that someone actually read the Bugzilla. They don't mark bugs fixed, or even that they've read them (changing the status from NEW to something else).

You mean that my feel about ARB and Khronos isn't low enough after seeing all this mess (that is seems to be supported by sole man, doing all this is his "spare time" and somewhat involved with mesa, which is not opengl only due to ARB's greedness)? Nice.



Been there, done that, have a public repository (https://bitbucket.org/alfonse/gl-xml-specs) for it. Oh, and I have an XML version of it too ;)
Yes, I've seen this repo already. Great work, BTW, still, I _hate_ XML. Well, maybe now I hate current specs more than XML, but now work is done and I'm already behind the schedule to migrate to something else.

Anyway, I've started to write down every bug I run across after I've seen your repo (with same file). Some of this bugs are intersecting with your "collection" - so, you mean that nobody really cares?

Alfonse Reinheart
02-04-2012, 12:58 PM
Anyway, I've started to write down every bug I run across after I've seen your repo (with same file).

That's one of the nice things about having a Mercurial repo: you don't have to "write down" bugs. Just change stuff and check it in. When GL 4.3 spec files come out, you just download them to a different branch and merge them with your bugfixes.


so, you mean that nobody really cares?

Cares about what?

The .spec files exist to produce headers for C applications (and by extension, all languages directly conversant with the C ABI). That's what the ARB wrote them for and that's how the ARB uses them.

So the ARB cares as long as their tool works for the purpose they intended it to work with.

The .spec files certainly can be used to produce bindings for other languages (though you'll need to use manual intervention for things like buffer objects and read pixels). But that's not their intent, so its no surprise that they don't work well for that purpose.

Simon Arbon
02-04-2012, 05:47 PM
The .spec files certainly can be used to produce bindings for other languages (though you'll need to use manual intervention for things like buffer objects and read pixels). But that's not their intent, so its no surprise that they don't work well for that purpose. It used to be, OpenGL was originally intended to be usable from any language that people choose to use.
The older .spec entries contain all of the information needed to generate bindings for any language, but at some stage the maintenance of this file was obviously taken over by a fundamentalist C programmer that couldn't care less about anybody that doesn't use his chosen language.
The recent entries are completely missing the type information required to properly use a strongly typed language.
The worst part is that the .spec file is completely inconsistent with itself, parts of the file written at different times have differences in syntax and layout.
In some entries, vital information on the size of data structures being used is appended to the end of the line as a comment!
I have spent days of my time over several months trying to sort this mess out, and am currently writing a program to parse the web pages of the SDK, as this seems to be the only source of the parts of the OpenGL API that are missing from the .spec and .h files.

Alfonse Reinheart
02-04-2012, 07:34 PM
but at some stage the maintenance of this file was obviously taken over by a fundamentalist C programmer that couldn't care less about anybody that doesn't use his chosen language.

That's one theory. It could also be that it was taken over by someone who suddenly had hundreds of functions and enumerators to add to the files and didn't think it was worth much of his time to write all the data in comprehensively.

I'm sure it makes you feel better thinking that this was done specifically to screw someone over, that it was a deliberate, malicious act. But isn't it a lot more likely that it was simply an oversight due to lack of time? It certainly makes more sense than imagining some raging C-nazi thinking, "Im in ur GL, scruin ur non-C lushors!!!11111!11!"

Maintaining this stuff takes time. And OpenGL is ultimately a volunteer effort. That means everything's not going to be perfect.


I have spent days of my time over several months trying to sort this mess out, and am currently writing a program to parse the web pages of the SDK, as this seems to be the only source of the parts of the OpenGL API that are missing from the .spec and .h files.

I find it interesting that three of us have these completely independent projects aimed at taming the .spec information. And none of us thought to ask anyone, "hey, could we pool resources? Let's get some e-mails or something together and see if we can't build something definitive." Even more interesting is that only one of us bothered to publish this information for others to use (unless you have a download somewhere that I'm not aware of).

The OpenGL community really needs to rally around something. The ARB can't really be relied upon to keep everything in a good form, so we need to.

If you're talking about parsing the man pages, you can just download them via SVN (http://www.opengl.org/wiki/Getting_started/XML_Toolchain_and_Man_Pages). They're in DocBook, which has a lot more semantic information than HTML.

I'm curious about what is missing from the .spec and .h files that can be found in the man pages.

Simon Arbon
02-04-2012, 11:22 PM
I find it interesting that three of us have these completely independent projects aimed at taming the .spec information. And none of us thought to ask anyone, "hey, could we pool resources? Let's get some e-mails or something together and see if we can't build something definitive."I would be happy to work together to create a better spec file.

Even more interesting is that only one of us bothered to publish this information for others to use (unless you have a download somewhere that I'm not aware of)At the moment i am still collecting information into a database which is then used to generate pascal code.
I cant generate a usable file until i have sorted out all the type conflicts, but i will publish it as soon as its working.
If others are interested in the raw information then i could write a function to output it in .spec or XML format.

Due to a hard-disk failure and some other urgent jobs i wont be able to work on this for at least 2 weeks, but I'll try and make it a priority after that.

If you're talking about parsing the man pages, you can just download them via SVN. They're in DocBook, which has a lot more semantic information than HTML.
I'm curious about what is missing from the .spec and .h files that can be found in the man pages.I am using the XML files from cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/docs/man/xhtml/ and pulling out the list of valid enums for each command.
The original .spec files divided the enums into groups with type names which were then used in the function definitions.
I am using the enum lists to create types for the new commands that don't have them.

Night Nord
02-05-2012, 04:34 AM
Maintaining this stuff takes time. And OpenGL is ultimately a volunteer effort. That means everything's not going to be perfect.

That's exactly the most suprising part of this all. I've always used to think that OpenGL is supported by big greedy consortium, that has power and money (mostly because mesa's non-OpenGL status). And now I see, that that's the same opensource stuff, but somehow bloated with trademark and patent problems. I've even though that maybe some kind of "LibreGL" may be worth doing, but it's highly unprobable that hardware vendors would support this =).



I find it interesting that three of us have these completely independent projects aimed at taming the .spec information. And none of us thought to ask anyone, "hey, could we pool resources? Let's get some e-mails or something together and see if we can't build something definitive." Even more interesting is that only one of us bothered to publish this information for others to use (unless you have a download somewhere that I'm not aware of).

Not everyone is working on open projects, you now. And there is much more projects (including GLEW, Unofficial OpenGL SDK, etc...) involed in spec parsing. GLEW, afaik, also parses textual specifications for specs (I've also has such plans - to extract documentation), and they are fairly open.



The OpenGL community really needs to rally around something. The ARB can't really be relied upon to keep everything in a good form, so we need to.

We need a "rally point" to rally upon. If not ARB, than who? If you want to make "better specs" (everyone want) than you need to make your own site with nice interface. Bitbucket,github,google-code repository, even perfectly googable, doesn't suffice.



I'm curious about what is missing from the .spec and .h files that can be found in the man pages.

Abstract type specifications (so non C/C++ wrappers could be built), exact enum usage groups (so C++ can separate them in different types, preventing user from supplying wrong enum into function), documentation.

I would be happy to cooperate. Code cooperation is useless at this point, as everyone seems to use different languages (lua, Perl, ?), but it would be useful to share opinions about what information should be kept and how it should be kept.

I believe, that all information about extensions and versions should be kept into RDB in group order version->extension->function->enum/type.

Version consists from extensions, one of which maybe version-extension (extension with same name as version itself, containing version's own functions). Extension consist from functions. Functions - from enums and types. Enums and types are kept in global flat tables enum->value and grouped by exported as desired (by using information from group). Types should be decomposed to floating-point/integer and size. Every level (version, extension, function, type) holds all related documentation.

Probably I'll find some time to implement example RDB within an upcoming week.

Alfonse Reinheart
02-05-2012, 09:59 AM
I am using the XML files from cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/docs/man/xhtml/ and pulling out the list of valid enums for each command.

Just be advised: those are not always accurate. Indeed, the 2.1 pages haven't been updated lately, and they won't be in the near future (http://www.khronos.org/bugzilla/show_bug.cgi?id=464). They've got a fair amount of bugs.

Granted, you don't really have much choice, other than manually walking the GL specification...


I've always used to think that OpenGL is supported by big greedy consortium, that has power and money (mostly because mesa's non-OpenGL status).

I don't know why licensing of the OpenGL name would imply "big greedy consortium". The original purpose of the license was to enforce compliance, such that an implementation that called itself "OpenGL" would have to be compliant with the GL test suite. Since there isn't a test suite anymore (though the ARB keeps telling us they're trying to make one), that sort of fell by the wayside.


And now I see, that that's the same opensource stuff, but somehow bloated with trademark and patent problems. I've even though that maybe some kind of "LibreGL" may be worth doing, but it's highly unprobable that hardware vendors would support this =).

It wouldn't matter, since graphics work is absolutely laden with patent issues. S3TC, anisotropic filtering, etc. At least with the ARB, we have an organization of many actual patent owners who agree to not sue each other or the ARB itself. And even with all of that, there are some who hold out patent claims against OpenGL.

You're really pointing the finger in the wrong direction here. The ARB is not the source of patent problems surrounding OpenGL.


We need a "rally point" to rally upon. If not ARB, than who? If you want to make "better specs" (everyone want) than you need to make your own site with nice interface. Bitbucket,github,google-code repository, even perfectly googable, doesn't suffice.

I'm not a web developer by trade, so I did the best I could with what exists. I don't generally have the time or inclination to make pretty websites.

I'm not against the idea. I just don't have the know-how or the ability to do it.

However, we don't need a website to rally around. We need people to rally around. People who are going to put forth effort to get certain things done. How that information is presented is certainly important, but not as important as getting that information out there. And generating that information is something we can work towards.

If none of us are good enough web developers to make a spiffy site, then so be it. But if I had to choose between the two, getting the information is more important than the website for it.


it would be useful to share opinions about what information should be kept and how it should be kept.

The thing that concerns me about a database is the fact that it won't be normative. The information will still ultimately come from ARB sources: the .spec files, the extension specs, etc. Because it's not normative, we need to be able to incorporate new information as the ARB produces it. When new extension specs and .spec files appear, we need to be able to pull it into the database.

And that's when having some form of text file in a repository would be important. Because then, we can use repository (Mercurial, Git, etc) merging and other tools to combine the ARB information with our own. There would be a paper trail of all of the changes we've made, and it would be clear when we've fixed a bug that the ARB picked up on and repaired.

Also, by using some kind of text file stored in a repo, we make it easy for others to access that repo, make bug fixes if they see something wrong, and then push those patches back to us for review and integration.

That's not to say that a DB can't be part of the solution. It would be great if someone pulled from the repo head and updated the database (and by someone, I of course mean an automated script) to match the head. But I think the DB should be secondary data (even if it's the one most people access), while the primary sources come from a repository that others can modify and propose patches for bug fixes.


I believe, that all information about extensions and versions should be kept into RDB in group order version->extension->function->enum/type.

I'm not a database programmer per-se, but I've done some personal DB work with SQLite3, so I understand the concepts of tables, relations, foreign keys to link elements to other elements, etc.

I'm not sure how what you're proposing fits into that structure.

The last database I designed (for a personal media library system), I designed it around how I planned to query for things. In terms of our needs, the database should make it easy to query for:

- All functions/enums that were added to a specific version of OpenGL. So you would ask for 2.1, and you would get only the functions that were added in the OpenGL 2.1 API. Items inherited from prior versions would not be included in the query. I found this quite useful in my GL loader.

- All functions/enums added to a specific version minus any functions/enums that were removed from OpenGL after some version. This one's pretty complicated. Basically, you're saying, "give me GL 1.2 stuff, but without the stuff that were removed in GL 3.1." It's an odd query, but I found it vital for building headers that contained only core stuff.

- Query the available versions.

- Query the versions where things were removed (currently just 3.1, but in the future there theoretically could be more).

- Query the parameters of a function.

- Query the enums that can be used for a specific function's parameter.

- Given an enum, query all of the function parameters that it can be used with.

This list doesn't include obvious things like "get types of function parameters/return types".

How does this list of queries fit into your proposed database structure?


And there is much more projects (including GLEW, Unofficial OpenGL SDK, etc...) involed in spec parsing.

Well, the SDK is mine, which actually uses my spec parsing for the GL Load module ;) But your point is made. And it's the same as mine: everyone keeps rolling up their own solutions because there isn't a centralized one.

Night Nord
02-05-2012, 10:54 AM
However, we don't need a website to rally around. We need people to rally around. People who are going to put forth effort to get certain things done. How that information is presented is certainly important, but not as important as getting that information out there. And generating that information is something we can work towards.


True. But, actually, there is little use of information if it's not presented well enough. Looking at your repo it's hard to tell if it is a complete work, work in progress and is it simply up-to-date. Also, I'm somewhat unsatisfied with current documentation and spec information - there is no way to see what extension to use for some purpose (there is always a bunch of extensions doing the same thing). After working with specs I've started to understand how it all works, but if _any_ opengl programmer should try to parse specs to get such information - that's a wrong way. Repo is good, but not enough for full-scale project. But, anyway, you're definitely right about that first we need good information and only then - way to present it.



And that's when having some form of text file in a repository would be important. Because then, we can use repository (Mercurial, Git, etc) merging and other tools to combine the ARB information with our own. There would be a paper trail of all of the changes we've made, and it would be clear when we've fixed a bug that the ARB picked up on and repaired.

Nice point. But, actually, I believe that database would be good in way that it maybe used to produce many text formats (XML, YAML, original spec "format"), including html documentation, C/C++ headers etc. So developers may actually choose which format they like most, while being sure that any format just present same information. Of course, it could be done by keeping all information in text file, too, but it will require parsing (and filling some internal structures) this text file prior to formatting new format, which could be counter-productive if we speak about online documentation, for example.

Also, if you keep all information in database, that you still may keep track of your fixes in git/hg/bzr repo within original specs and just reimport all related stuff in database on update. But database may keep more information than it's available from specs (including manually added information), which wouldn't be affected on reimport. That's much harded to achieve by storing information in some text-format. And it would be just bloated with different information and, therefore, unmaintainable.

And about SQL queries (not tested actually, not sure about right order in LEFT JOIN's):

- All functions/enums that were added to a specific version of OpenGL.

<div class="ubbcode-block"><div class="ubbcode-header">Sql Query:</div><div class="ubbcode-body ubbcode-pre" ><pre>
SELECT function.name FROM version
LEFT JOIN extension ON ((version.id == extension.version) AND
(extension.name == version.name))
LEFT JOIN function ON (extension.id == function.extension)
WHERE (version.name == "VERSION_2_1");
[/CODE]


All functions/enums added to a specific version minus any functions/enums that were removed from OpenGL after some version
Just not query and append results from VERSION_x_x_DEPRECATED extension. That is - we just keep deprecated stuff in separate extension with _DEPRECATED prefix. If we need both - query for both. That's how I'm doing it now ;)

Well, probably string suffix is not so much RDB'ish, so we may add bool flag "deprecated" in each extension's row.


Query the available versions.
That's simple. <div class="ubbcode-block"><div class="ubbcode-header">Sql Query:</div><div class="ubbcode-body ubbcode-pre" ><pre>SELECT version.name FROM version;[/CODE]


Query the versions where things were removed (currently just 3.1, but in the future there theoretically could be more).
Query all versions that contain extensions with _DEPRECATED suffix/"deprecated" flag set to 1? That's also simple (too lazy to write sq-queries without real database at hand)


Query the parameters of a function.
<div class="ubbcode-block"><div class="ubbcode-header">Sql Query:</div><div class="ubbcode-body ubbcode-pre" ><pre>
SELECT function_params.name FROM function
LEFT JOIN function_params ON (function.id = function_params.function)
WHERE function.name = "Vertex3f";
[/CODE]


Query the enums that can be used for a specific function's parameter.
If we would have such information (afaik, there is no way to get such information without parsing textual specifications), this will add additional param->type->enum mapping in scheme.
<div class="ubbcode-block"><div class="ubbcode-header">Sql Query:</div><div class="ubbcode-body ubbcode-pre" ><pre>
SELECT enum.name FROM function
LEFT JOIN function_param ON (function.id == function_param.function)
LEFT JOIN type ((function_param.type == type.id) AND
(type.enumerated == 1))
LEFT JOIN type_enum ON (type_enum.type == type.id)
LEFT JOIN enum ON (type_enum.enum == enum.id)
WHERE function.name == "GetDoublev" AND function_param.position == 1;
[/CODE]
Probably it's incorrect, but that's possible (by using separate mapping tables, of course).


Well, the SDK is mine, which actually uses my spec parsing for the GL Load module
So, that's your idea (about keeping functions in separate headers and namespacing gl/glx functions) I've shamelessly stoled =)


And it's the same as mine: everyone keeps rolling up their own solutions because there isn't a centralized one.
Well, we just can't use external libraries without getting terrible headache, so I've thought that gl wrapper should be rather easy to reinvent.

nigels
03-04-2012, 09:18 AM
Chiming in here as the GLEW maintainer.
http://glew.sourceforge.net/

GLEW uses perl scripts to parse the extension registry specifications, rather than the .spec files. It has an intermediate format in auto/extensions which is easily parsed, which ought to be suitable for import or cross-checking with a database or XML scheme.

My own impression is that Khronos .spec maintainers would be supportive of an alternative scheme, providing that there are a set of tools for easily converting and maintaining the central specs.

My own modest goals for GLEW include improving the toolset, converting the perl scripts to python, and importing .spec files as an alternative and/or cross-check for the extension specification import. I figure an extension spec to .spec file converter might be a useful component too.

- Nigel

Alfonse Reinheart
03-04-2012, 10:17 AM
The problem is that nobody seems willing to work together, pool resources and get it done. We all have our own partial implementations, which we use to gather information for our own projects.

Each of us has our own little projects, but none of us is willing to take what the others offer. And in some cases, some of us are keeping the information private.

The only way for this to move forward is for several people to set aside their personal pet peeves about the way things ought to be done (no XML, we should use a remote DB, etc) and say, "I want to get the information out there in some form. Let's get it done." And then pool their resources to get it done.

Thus far, this community isn't doing that. We're just a bunch of people looking at a mess, no one person capable of cleaning it up, but the group still stands around expecting someone else to do it. If we worked together, we'd have it finished by now, but we constantly and consistently refuse to.

I'd be willing to join such an effort. But I can't do it alone.

nigels
03-04-2012, 10:37 AM
Alfonse,

You make a good point. What I do for GLEW is for GLEW purposes primarily.

I do occasionally push spec changes back to the official registry, for the broader good, but I have not paid particular attention to the .spec files.

What I'd point out about "getting it done" is that OpenGL is a moving target, it's not just a matter of getting everything correct(*) at this point in time. It needs to be a suitable form for longer term maintenance - one year, two years and five years from now. And that I think is a serious challenge.

GLEW itself is showing signs of age, and even though I maintain it, I don't like dealing with perl.

One simple suggestion I have is for there to be a community maintained git repository of the .spec files, along with a branch containing fixes that are generally agreed-upon. Not concerned with conversion scripts, GL loaders, etc, but simply a fork of the .spec files (and perhaps the extension registry) for sharing our patches easily. From a GLEW perspective, I'd be happy to have an alternative extension registry to use, as long as it was kept up-to date.

- Nigel

(*) Correct in terms of the OpenGL implementations in widespread use.

Alfonse Reinheart
03-04-2012, 11:03 AM
Correct in terms of the OpenGL implementations in widespread use.

I'm not sure what you mean by that. The source of this information is the specifications; that's what defines correctness.


It needs to be a suitable form for longer term maintenance - one year, two years and five years from now. And that I think is a serious challenge.

Why? Text is text. If you have a parsing tool that can read it, and a reasonably simple format, then you have something that works.

What specific issues are you referring to?


One simple suggestion I have is for there to be a community maintained git repository of the .spec files, along with a branch containing fixes that are generally agreed-upon.

Been there, done that. (https://bitbucket.org/alfonse/gl-xml-specs) The main branch contains the fixes, but they're imported from the "spec-mods" branch, which is where they were first entered.

When the .spec files are updated externally (by the ARB), I simply pull those into the "spec file update" tag, then update the tag to match, then merge that with "spec-mods", and merge that back into the default branch.

What else you got?

nigels
03-04-2012, 11:47 AM
Been there, done that. (https://bitbucket.org/alfonse/gl-xml-specs) The main branch contains the fixes, but they're imported from the "spec-mods" branch, which is where they were first entered.

When the .spec files are updated externally (by the ARB), I simply pull those into the "spec file update" tag, then update the tag to match, then merge that with "spec-mods", and merge that back into the default branch.


Excellent. So between GLEW and your projects - can we agree to move that work to a separate project/repo thats a shared area purely concerned with tracking the opengl.org updates and rebasing the appropriate patches? In addition to the .spec files, I'd be interested in a similar scheme for the extension specs.

We'd have to agree on a revision control system, my preference would be git.

And as a separate question - is there a known Python parser for the .spec files?

- Nigel

Alfonse Reinheart
03-04-2012, 01:03 PM
Here's the thing.

I don't want this to happen piecemeal. I don't want to just create a little repo somewhere that contains the .spec files with some bug fixes. I already have that, and that repo is free for anyone to use and do with as they see fit.

If you want me to break my oath to never, ever install, learn, and/or use Git without being paid to do so, then I want to see something more out of it than a copy of what I already have. I want to see that place become the central hub for not just storing the .spec files, but for transforming them into other formats. For processing them and using them to do actual work.

I want to see a place where there are scripts that can process the .spec files, convert them into any number of formats, and then do whatever with them. I want to see the .spec files not become the storage for information; I want to see them simply be how the ARB drops spec information onto us, and we integrate that into our preferred storage medium, which contains more information than the .spec files can actually store.

In short: I want something real, not just a little toy project that nobody knows about and maybe a couple of people benefit from.

Either way, you're free to pull from my repo, convert it to Git using some tool, and stick it in a Git repo somewhere. But if you want my time and help in building and maintaining this thing, then I want the goal of this project to be something more than just a place for us to fix bugs in the .spec files. I want us to decide to design what we want to put together, figure out the best way to do it, and then get it done.

That's the price for making me use Git ;)

nigels
03-04-2012, 07:06 PM
Alfonse,

Sorry about mentioning git. I don't have anything against mercurial, except that I've never used it. My general impression is that git is more popular overall, but let's not get sidetracked with a git vs svn vs hg vs etc, etc...

The reason I propose 'just' the .spec and extension files is that it ought to be neutral of everything downstream - XML, YAML, perl, python, C, C++, GLEW, glee etc etc. I don't want it seen as a for-GLEW-purposes kind of thing. Maintaining a list of downstream tools and applications ought to suffice, rather than trying to be a one-stop shop.

It's less ambitious than telling Khronos they ought to replace .spec files with something else. But it ought to be supportive and complimentary to such an effort.

What I'm a little more specifically interested in doing is a path from .spec to GLEW, for cross-checking and correctness of GLEW. And you're right to point out that your fork of the .spec files is suitable for doing that as it is.

It would be a bit of shame if the main blocker was agreeing on a revision control system.

- Nigel

nigels
03-04-2012, 10:40 PM
I came across this, which has some potential, from a "GLEW redone in Python" perspective.

https://github.com/AlexandreFournier/gl-spec-parser

- Nigel

obirsoy
03-05-2012, 08:06 AM
There is also this one (i.e. using python to parse .spec files)
https://bitbucket.org/yotis/glxx/wiki/Home

nigels
03-05-2012, 08:21 AM
Nice one. Thanks a bundle.

- Nigel