Feedback on new 3D mesh file format (OpenCTM)

Hi!

I know this a bit off topic, and a bit long, but I hope to get some response anyway…

I have created a new file format for storing 3D triangle meshes, and I would like to get some feedback from potential users before going beta.

Why another one? Well, this one is compressed, so it can store most meshes in a very dense format, compared to other similar file formats. Also, it is completely free and open, and comes with a software library that makes it easy for most applications to load and save files in the new format (which is called OpenCTM, by the way, for “Compressed Triangle Mesh”).

There are two things that I would need some help with right now, though:

1. Feedback of how useful this format would be for Your applications, and if it isn’t, how to alter it to make it more useful (i.e. what is missing?). This can be anything from compiler/language support, to missing features in the file format and lack of support in major 3D applications.

2. I am trying to build a test set (i.e. a bunch of 3D meshes/models) so that I can test and optimize the software library. So far I have worked mostly with the Stanford bunny and some custom simple meshes, but I am looking for some more real-world meshes (as in used in actual 3D applications). I would like OpenCTM to support CAD models, scanned models, game models and art/movie models, in all sizes. The models would have to be free / public domain, of course, and preferrably in a format that Blender can load.

Feel free to try the software out if you wish, but please bare in mind that the project is in alpha stage, which means that there are no tutorials, no “getting started” documents, and both the file format and the API is likely to change before the final release.

Some quick links:

OpenCTM home page
Feedback forum
API docs
Source code (browse SVN repo)

The goal is, of course, that OpenCTM will be a useful file format, and a useful tool. So I hope that you will find it interesting enough to participate in its development to make it as great as possible…

is this an ascii file format ?
or you using just binary (which i guess can lead to endian problems on different systems) ?

I looked at the raw-type file-format, and find it way too simple.
A file contains only one mesh? No material-data for this mesh, either? One set of texcoords? No vertex-maps (putting weight/color/morph/whatever) ?

It’s a shader-world. Apart from 1 vec3 for position, 1 vec3 for normal and several vec2 for ST coords, there are multiple per-vertex attributes, all custom- and only your custom shader knows what to do with them.

Here’s a simplistic intermediate format I use:

//==============[ model structures ]=====================================================[
class XSurface{
public:
	char* Name; // i.e "blah $MaterialTex2$cubemap1=woglinde04$someOtherparam=etc"
	vec3  baseColor;
	bool  IsDoubleSided;
	bool  IsSmooth;
	
	float luminosity;
	float diffuse;
	float specularity;
	float glossiness;
	float reflection;
	float transparency;
	float refractionIdx;
	float translucency;
	float bumpIntensity;

	int	  NumVerts; 	vec3* AllVerts; // AllVerts[NumVerts];
						vec3* AllNorms;	// AllNorms[NumVerts];

	int	  NumQuads;		int*  AllQuads; // AllQuads[NumQuads*4]
	int   NumTris;		int*  AllTris;  // AllTris[NumTris*3]
	int   NumLines;		int*  AllLines; // AllLines[NumLines*2]
	int   NumPoints;	int*  AllPoints;// AllPoints[NumPoints*1]

	vec3 AABBmin; // bounding-box start
	vec3 AABBmax; // bounding-box end
	float BoundSphereRadius; // bounding-sphere radius

	
	
	typedef struct{
		int		Type; // 'VCOL', 'RGB','RGBA', 'WGHT'
		char*	Name;
		vec4*	Values; // Values[NumVerts]
	}VTXMAP;

	typedef struct{
		char*	TexFileName;
		char*	VMapName; // must exist in AllVtxMaps[]
		int		Channel; // 'COLR', 'DIFF','LUMI','SPEC','BUMP'
		float	Opacity;
	}UVMAP;

	int   NumVtxMaps;	VTXMAP*	AllVtxMaps; // AllVtxMaps[NumVtxMaps];
	int	  NumUVMaps;	UVMAP*  AllUVMaps;  // AllUVMaps[NumUVMaps];


	vec4* GetUVChan(int ChanType,int order,UVMAP** ppvoutUV=null,VTXMAP** ppvoutVTX=null); // if failed, prints info and calls ExitProcess
	vec4* GetVTXMap(int MapType,const char* name,VTXMAP** ppvoutVTX=null);
	char* GetUVChanTexture(int ChanType,int order);

	void ConvertQuadsToTris();
	vec3* ComputeNormals();
	void MakeTrianglesBeFlat(); // reorders indices of triangles so that the first vertex's normal is 

	XSurface(){
		memclear(this,sizeof(XSurface));
	}
	

	~XSurface(){
		xfree(Name);
		
		for(int i=0;i<NumVtxMaps;i++){
			xfree(AllVtxMaps[i].Name);
			xfree(AllVtxMaps[i].Values);
		}
		for(int i=0;i<NumUVMaps;i++){
			xfree(AllUVMaps[i].TexFileName);
			xfree(AllUVMaps[i].VMapName);
		}


		xfree(AllVerts);	xfree(AllNorms);
		xfree(AllPoints);
		xfree(AllLines);
		xfree(AllTris);
		xfree(AllQuads);

		xfree(AllVtxMaps);
		xfree(AllUVMaps);
	}
};


class XModel{
public:
	ObjVector meshes; // type is XSurface*;
	ObjVector bones;
	
	XModel(){}
	~XModel(){
		foreach(&meshes,XSurface*)delete edx;
	}
};

XModel* LoadLWO(const char* FileName);

This is fed to a material’s Transcode() method to produce a packed file.
Here’s the vertex-structure for some material:

struct VTXDCL_Type2{
	vec3 pos; // attrib0
	vec3 norm; // a1
	vec4 uv0_uv1; // a2: tex coords #0 and #1
	DWORD colorMul; //a3
	DWORD colorAdd; // a4
};

Here’s an example vtx-struct of a material for wind-affected vegetation, where attribs are somewhat compressed:

struct VTXDCL_...{
	char pos[4]; // a0: byte-sized xyz and scale
	short uv0[2]; // a1
	char norm[3]; char windEffectiveness; // a2
};

Simply, multiple named vertex-maps with size vec4 are a must-have, and then having per-material transcoders are needed. And most importantly, having multiple meshes, with different materials (and same materials but with different settings) are a must-have.

dukey:
Yes, all formats are binary - quite unavoidable to keep the size down. However, portable endian handling is central in the format & API design.

Ilian Dinev:
You touch a very important issue: the file format deals ONLY with the triangle mesh data, and it only supports a single object (i.e. a mesh where all the triangles share the same transformation, material, etc). It is not a scene graph, it is not a full blown model description format.

For providing a full model/scene description, I envision a wrapper file format that goes all the way with materials, physical properties, animation information, transformation, linking, etc. etc. Much like the open document format and other similar formats, that format would typically be a ZIP archive containing an XML-file (for the scene/model/material/etc-description) and a bunch of binary OpenCTM files that are referenced from the XML file.

The making of such a file format is another project, and I do not feel experienced enough to make it myself right now - typically we are talking about tight integration with existing 3D modeling tools etc.

Anyone interested?

Ilian Dinev:
I agree that in order to be useful for shader-centric productions (games etc), you probably need multiple texture coordinates and per-vertex attributes.

Just so that I understand this correctly… When you say vertex maps, you mean named arrays of per-vertex attributes, with user selectable types (e.g. int, float, vec2, vec3, etc)?

So if you agree with the discussion about the single mesh only philosophy above, would you agree that the following are valuable additions to the file format? :

• Support for 0…N named per-vertex texture coordinates (vec2).
• Support for 0…M named per-vertex attributes (any type).

…separating the two is good for at least two reasons:

1. Data can be compressed better when you know its properties (e.g. texture coordinates usually have some relationship to the point coordinates, and are typically in the range [0,1]).

2. It is easier for a generic modeling package to understand specific attributes (e.g. Blender has a concept of UV maps), so if all applications are forced to speak the same language, data interchangeability is improved.

Portable endianness? Then you’ll also have to make sure datatypes are properly aligned (ARM cpus issue an exception otherwise)? And let the artist write XML instead of doing it all in the modeler of choice :)? Then torture the cpu in parsing that xml…
Maya and 3dsmax file-formats are unparsable, probably same for Cinema4D and XSI. Devs use custom modeler-specific scripts to export models. Mesh/material-type specific scripts. LWOs are readable, but impose some limits (clamp values to 0…1) when drawing vertex-colors/weights (fixable via scripts). OBJ/3DS are the extreme of limitations. Other file-formats are also limited and obscure. Collada makes you take a coffee-break to see your model ingame, to see which of your stuff wasn’t exported or imported correctly; after a few coffee-breaks your HDD fails.

Anyway, you might like to read on the java M3D file: http://www.j2medev.com/api/jsr184/file-format.html
It implements things nicely for fixed-func stuff. The tried and true data-types and management that ancient stiff rasterizers handled, to draw ancient graphics. But now we have an extremely flexible system with GL3. Exotic attribute data per vertex, used by shaders to compute color, were just the start years ago. Right now we also use per-primitive, per-adjacency, per-dynamic geometry, per instance, per pass data; with texture-arrays and 1D/2D/3D input-sources galore. Want to mix-in raytraced spheres in the scene - you can; want to use dual-quaternion transformation - again you can; want to optimally draw neon tubes as boxes for deferred rendering - you can; want to draw the same mesh in several passes, with different sets of data and shaders - you can. But a standard that dictates from the start what is what - will only get in the way, and thus be thrown in the Recycle Bin sooner or later. Unless you develop PlayStation1 - class graphics, that is.

A programmer should only need to press F5 to build and see his app working; an artist should at most need to press Ctrl+S, Alt+Tab to preview the model - imho. So, writing XML, packing files, checking the XML is in sync with what was drawn, etc etc - is out of the question. It’s like making a programmer draw a portrait after every function he implements. This brings us to automation:
Every modeler app has its peculiar way of handling artist-drawn data, so there are 3 options: 1) limit to a common denominator; 2) implement thorough data-listing that supports all 10 major CAD apps; 3) implement thorough data-listing that supports the CAD your artists use. Obviously 3) is the best way to quickly/easily/sanely create and research modern graphics, and its developers use it to make commercial games with it thus keep the scripts/importers in-house.

Sorry to shoot down your idea, I just want to share my view of how diverse and flexible rasterizing has become - that a file-format should either be intermediate (and actually has no place on the HDD) or contain baked data, that is extremely easy and fast to parse and upload to VRAM, in a customizable format, bound to specific custom materials and vertex-declarations (thus has no reason to be a standard).

In case you go ahead, you’ll initially support that limited pos/norm/uv dataset, with transform info. Then, by request you’ll start adding random features from modelers people want supported (apart from learning their modelers and implementing exporters for said modelers). It’ll gradually increase complexity to become a possible mess. Artists will wait more and more for their stuff to be added as an extension, and then wait more - for someone to write code to visualize that extended data. A programmer will have to wonder how the artist intended the given mesh-parts with all those attached extension data-blobs should look/be, and start creating a system of dynamically generated shaders that can only hope to approximate the intent. Until, I guess, it all crumbles onto itself. That is, provided there’s active interest in the file-format.

Yes, that looks quite identical to the simplistic XSurface struct I currently use. Except that support for anything but vec4 for named attribs is unnecessary clutter: specifying an integer > 2^24 is kind of unthinkable (a float keeps an integer < 2^24 perfectly intact); specifying just a float/vec2/vec3 clutters the transcoder’s code; specifying a mat4 attribute per vertex is impossible to do by the artist in any modeler anyway.
Yet, that XSurface struct is an intermediate thorough inflated data-listing, that has no place on the HDD; it should only serve as input to a transcoder.

P.S. I have not published even one shader-based commercial game (only fixed-func or custom SW rasterized comm ones); I base my posts on experience, personal KB, constant research on modern graphics, spying on methods used in modern games, modeling all art myself and searching for the best way to speed-up development for indies (myself).
I’ve found the XSurface and stuff to be over-satisfactory. Multiple-passes, instancing, custom object/light stuff expressed by primitives, etc etc: it’s all handled. It just needs a transcoding, a loading, and one or more drawing funcs per material - code that I would have anyway had to write.

Yes Ilian, I actually agree with you fully. This is why I also left out the material and scene description information completely - it IS heavily dependent on which application you are using.

The ZIP+XML+OpenCTM-format that I was talking about is actually what I think every application/programmer/game will do on their own, in order to be as flexible as possible. Just look at VRML. Who is using it?

I think that in general it is better to be limited than to try to be flexible and fail…

So, despite this, do you see any purpose or gain of having a format like OpenCTM in your productions (or similar products)?

I know of some areas where it definitely would be useful (CAD/CAM, 3D scanning, archiving, etc), but it would be nice if it could be useful for other areas as well (without changing the concept too much).

Except that support for anything but vec4 for named attribs is unnecessary clutter…

I agree - in general a vec4 will compress to the same size as a single float, as long as the other elements are all zero, so from a size point of view, it’s no bloat. It might be an issue for memory consumption and/or the API (I’m thinking embedded systems here), but I really like simplistic solutions.

One area where such a file-format could be useful is: pack some parts of a mesh from one modeler, import them in another modeler (or special tool), do stuff only the second modeler/tool can do, and then readback modified stuff to the initial modeler. I.e:

• Calculating UV-unwraps in a special/custom tool.
• Drawing vertex-colors/weights in a better/custom tool, or texture-paint, or sculpting

At the minimum, it can provide an intermediate file-format to quick-start development with. And have API to do the custom packing (transcoding) of a mesh with few lines of code.

Ok, so I am currently rewriting the API and file format slightly, to support variable number of texture coordinate sets and vertex attribute sets. I think it’s a good idea, and it actually cleans up the software library a bit.

Now, I still have not gotten any feedback regarding actual 3D models… Does anyone have any good models for a test set that I can use??

There are a bunch of models on the net, made by real artists. Import in Blender, convert to ply. Then learn the script of your fav modeler and write scriptcode to export to CTM.

I currently work with Blender, and export PLY (it’s quite modular and easy to parse). Then I convert from PLY to CTM (and back).

I have found the following resources:

• The Stanford 3D scanning repository (the bunny, dragon, Buddha etc).
• The Blender open movies (Elephants Dream and Big Buck Bunny) contain a bunch of interesting models.
• Various free, low quality models on various sites.

Any other recommendations?

I have a hard time understanding why the need for specialist built in compression, coming from a unix background we decided that instead of having file formats with compression built in - compression would be applied to files.

eg: you have an xml file called ‘foo.xml’ you want to compress that xml file you would compress it with the ‘gzip’ utility to produce ‘foo.xml.gz’. you can even write parsers that can dynamically de-compress as is needed, for example on most linux distros there is the ‘zcat’ application that will read a .gz file and display the de-compressed text as its needed.

so basically, why the need for an entire file format when i could just take a foo.collada file and gzip it producing foo.collada.gz, sure a specialist compression technique could squeese a few more bytes out but its not really going to matter much.

GordAllot,

Here comes the beauty of it all: Simply trying to gzip a geometry file will only give you some compression (usually in the order of 50%). The problem is that geometry data is A) Very unstructured with few similarities, and B) based on floating point binary data that has almost no reoccurring binary parts.

Example (Stanford Bunny, preliminary results):

Raw data (OpenCTM RAW format, without compression - similar to e.g. to the .3DS format):
1.20 MB

Same file, in a ZIP archive:
660 KB

OpenCTM MG1 compression (some data mangling, lossless):
454 KB

OpenCTM MG2 compression (heavy data mangling, fixed point):
193 KB

The key here is to apply “entropy reduction”, based on what we know about the data.

It is very similar to what PNG and JPEG does. E.g. try to gzip a .BMP file and see if it will be as small as the corresponding .PNG file, which also uses gzip internally, by the way. PNG knows that it is dealing with image data, and tries to represent the data in a more clever way (just store what is actually needed).

OpenCTM does exactly the same, but for 3D triangle geometries.

Is it worth the effort? For some applications, no, for other applications, yes. For instance, where bandwidth and/or storage is limited, a factor 5x to 10x in compression can make a huge difference.

Hi again!

I have updated the API and the file format to support multiple texture maps and multiple custom vertex attributes.

Do anyone have any opinions about the new solution? Specifically, is the corresponding API functionality useful, and is there anything more that would be necessary (e.g. file name references for the texture maps?).

Interested persons, please have a look at http://openctm.sourceforge.net/apidocs .