I looked into Collada, .x, and a few other file formats, and couldn’t find a fast-loading file format with support for hierarchal meshes and skinned animation. So I wrote my own.
The idea is that you can load this straight into your engine, without any of the fiddling around that other formats require. Tangent-bitanget arrays are pre-calculated, animation keys are already interpolated at regular intervals, and the data can be read straight into vertex arrays to be sent to the GPU. It’s easy to read and can be expanded to support new data without breaking old loaders.
At this point all I have is a .b3d to .gmf converter, but I am talking to a few people about adding exporters for various 3D packages once the file spec is finalized.
This isn’t just vaporware, I have animated skinned models working right now:
Indeed. I evaluated tons of them and could not find a single one that supported skinned animation and hierarchies, and didn’t require tons of preprocessing.
BTW, Collada is not meant to be a final format. It is an interchange format. It is intended to represent 90% of the data in a tool like Max or Maya. It is for toolchain work; it makes it easier to switch between tools like Max and Maya. It also makes it easier to write converters that go from a tool to the end format that you want.
And I have to agree with Knackered. The world doesn’t need another geometry file format. You may find it useful, but I wouldn’t expect it to be considered generally useful. In my experience, the final geometry format should be designed based on need anyway.
A simple format like that isn’t a bad idea. Many small projects don’t have the time to implement universal format like collada. Unfortunately the known formats that supports bones aren’t very GPU friendly. For example MD5 and cal3d are designed for CPU based animation.
So a simple GPU friendly format could be helpful for small projects, if exporters will be available.
From my experience, portability and performance are mutually exclusive. You always end up with some more or less internal binary format that suits your own needs. I’m no game developer, but I guess that this is especially true for game data, hm?
If you like it, use it and give me feedback. If you don’t, I don’t care. It boggles my mind that there is not one widely-supported model format that simply stores the data the way you need to store and upload it to the GPU; I don’t believe in the idea that we are all doing such different things that no model format can be shared; We’re all just pushing matrices and uploading vertex arrays, and I purposely design the vertex array format so it could be suited for any use.
Believe me, if there was another file format like this out there I would have gladly used it!
It boggles my mind that there is not one widely-supported model format that simply stores the data the way you need to store and upload it to the GPU
That’s because there’s no one “way you need to store it”. There are a number of different ways to store data, and the particular situation determines how this data gets stored.
We’re all just pushing matrices and uploading vertex arrays
Except that some of us want to be able to change texture coordinates by substituting one set of texture coordinate mappings with another. Or a different color array (for having different sides in an RTS). Or the z-component of the position.
The number of bones that the user expects to upload for rendering a piece of a mesh is an implicit part of the model format, because it must inform the process of creating that mesh. If your shader can only index 32 bones, but you use 64 in your mesh creation, you have a broken mesh.
We are not all just doing the same thing. Not everyone is doing something as simple as “push matrix, upload some verts.”
You can store up to 255 bones if you use bytes for the variable type, and if you really wanted to you could use words or integers.
Most engines will support much fewer than 255 bones. In my own engine, I just have an error message pop up if the number of bones exceeds the engine’s limit, and I just tell my users they need to keep it under a certain threshold.
Most engines will support much fewer than 255 bones.
Actually, most engines of note will take whatever the shader-defined matrix limits are and break your skinned mesh up into bit-sized pieces that use no more than that number. At render time, it then puts these pieces back together.
What about the Ogre format? It is probably alright.
For a while I was working on taking their code and modifying it to my liking but I suddenly stopped.
Is their format documented? I think I looked at it a while ago and couldn’t find a spec.
I think I will move things like material name and limb names into a custom properties block. It would just be a list of keynames and values, like this:
“name”=“biped_leg01”
“material”=“zombie.mat”
I think the really important data is the matrix hierarchy, mesh data, animations, and weighting. Everything else is just fluff that may vary depending on the implementation.
Also going to replace the position/quat stuff with local 4x4 matrices, because I think that is less ambiguous.
I’ll make the exporter sources open and free so people can modify and add to them as they see fit. As it is now, you can have things like multiple texcoord sets and you can expand the format without breaking existing loaders.
The thing I have had problems with was getting animated skinned mesh data from modeling packages to my engine, so I think if that core functionality is provided, this can help a lot of people. Then they can modify the exporters however they see fit to add their own functionality.
Good for you, I agree collada isn’t straight forward, and frequently you need something that “just works”. I looked at your spec, and quickly decided it wasn’t for me,I need more supporting documetation such as quick referances, example code, etc… to me there is no payoff for using what you did. Don’t get me wrong, I would probably use it if it had good documentation or looked like it would “just work” for me.
In summary, from an outsiders perspective, your spec and collada are the same, no friendly documentation. Collada is more advance but in the end it, the only good tools for it seem to be a pay to play solution. Don’t let the flametards get to you, remeber the people running collada are now running opengl, for better or worse.
Thanks for the feedback. I am still fiddling around with this, but I will get some example code up in the next couple weeks. I think it needs an OpenGL example that loads the model and displays animation.
Collada is an interchange format, so I don’t consider the two formats to be related. In fact a Collada -> GMF converter might be a good idea, as it would allow support for any modeling package that properly exports a Collada file.
When I brought up the idea of a game-ready format on the Collada forum, the attitude seemed to be that everyone on the planet requires their own unique file format, and needs to write their own exporters. I don’t buy that. I can at least provide a good easy-to-read animated file format, along with plugin and converter source code so the format can be extended according to one’s own needs.
Whether you buy it or not, it’s basically true. I’ve given examples of the variety of stuff that some users will want that others will consider (at best) cruft.
Now, if you want a “hobbiest” format like .X files, which are useful for getting something up and running, but useless in professional code, that’s one thing. A good hobbiest toolpath going from Collada to a game mesh loader/renderer wouldn’t be a bad idea.
You sure make a lot of effort to subtly disparage my work. You are quick to label it as a “hobbyist” project, and as “useless for professional code”, and you promote nonsense like the idea of a game using more than 255 bones in a model, as if you are the expert on all these subjects. Then you quickly take the opportunity to promote Collada for my clearly inferior format, since I don’t have a 100+ page spec or a bloated SDK that gets completely rewritten every six months. At the same time, I have made no comment on Collada other than agreeing it was an interchange format, and was not designed for loading directly into a game.
Thank you for helping me to see the error of my ways, I was under the impression that loading vertex data straight into memory and to the GPU would be faster, but clearly it would be better to do things your way.
Thank you for barging in and derailing this discussion you claim to have no interest in. If you truly had no interest in this, I think you would just not bother to comment so many times. I am not selling anything or forcing you to do things my way, I am only asking for ideas and putting out some free tools, which people may take or leave as they see fit.
It took a lot of work to implement this, and I thought it might be helpful to others, but clearly that idea does not sit well with you.
I’ve read your spec, and I actually think it’s quite nice that someone’s formalised something similar to our in-house file format. If you produce a max and maya exporter I’ll add support for it to my renderer.
Just a little criticism, you should support interleaved vertex formats somehow, I wouldn’t want to be interleaving every load.
Yes, you’d probably just introduce another sub-chunk type to the vertexarray chunk in your system. Something like this:
GMF_VERTEXARRAY
GMF_VERTEXATTRIB
long ;data type (GMF_POSITION, GMF_NORMAL,
GMF_TEXTURE_COORD, GMF_COLOR,
GMF_TANGENT, GMF_BITANGENT,
GMF_BONEINDICE, or GMF_BONEWEIGHT)
long ;type (GMF_BYTE, GMF_UNSIGNED_BYTE,
GMF_SHORT, GMF_UNSIGNED_SHORT, GMF_INT,
GMF_UNSIGNED_INT, or GMF_DOUBLE)
long ;number of elements
long ;offset in bytes from data origin
long ;number of vertices
long ;vertex stride
data[n] ;vertex data.
You’ll need the vertex stride to support padding schemes, and maybe if it’s 0 the stride would default to the total byte footprint of all attribs.