Decent Style Observer

I am trying to make a decent style observer to skoot around my 3d World. I am using DirectInput for the mouse(but that should not matter). I can look around fine from the origin (up, down, left and right) but when i go to move in the direction i am facing, it flips out. I have peiced together a way to do this with gluLookAt, but i would rather just build the matrix my self. The up-vector is always changing as well and so I figure that I should just build it myself anyway.

this is how I am doing it so far:

mat = translation3D(vec(x,y,z));
mat = mat * rotation3D(vec3(0,1,0), theta * 3.1415926535898 / 180);
mat = mat * rotation3D(vec3(1,0,0), -phi * 3.14159265358 / 180);
mat = mat.inverse();
mat = mat.transpose();
float Vmat[16];
// Turn mat into OpenGL friendly Matrix Array
Vmat[0] = mat.v[0].n[0]; Vmat[1] = mat.v[0].n[1]; Vmat[2] = mat.v[0].n[2]; Vmat[3] = mat.v[0].n[3];
Vmat[4] = mat.v[1].n[0]; Vmat[5] = mat.v[1].n[1]; Vmat[6] = mat.v[1].n[2]; Vmat[7] = mat.v[1].n[3];
Vmat[8] = mat.v[2].n[0]; Vmat[9] = mat.v[2].n[1]; Vmat[10]= mat.v[2].n[2]; Vmat[11]= mat.v[2].n[3];
Vmat[12]= mat.v[3].n[0]; Vmat[13]= mat.v[3].n[1]; Vmat[14]= mat.v[3].n[2]; Vmat[15]= mat.v[3].n[3];
glMultMatrixf(Vmat);

so i do a translation then my rotations, but this does not work properly…

Any suggestions?

Take the dir vector and add it to pos vector.
pos.x +=dir.x *speed
pos.y +=dir.y *speed
pos.z +=dir.z *speed

Using matrix:
struct {
VECTOR right;
float filler1;
VECTOR up;
float filler2;
VECTOR dir;
float filler3;
VECTOR pos;
float filler4;} MATRIX;

Hope that helps.

I will look at that aaaaand http://romka.demonews.com/opengl/demos/other_eng.htm
gosh!
this guy is good, i also reccomend the text lib he has!!! wow!

Hehe, you could also just take a peek at the Descent source code.