PDA

View Full Version : Quadro page faulting for 5 seconds all the time



peterfilm
03-10-2010, 09:17 AM
i can report serious stuttering on the quadro line of cards at the moment. This is most evident under extremely heavy API load. In other words, when lots of uniforms are being updated for many batches, when the batches are in display lists. When I say serious, I mean 5 second stalls when the load suddenly increases (i.e. when you turn the camera to face the model and there's a sudden jump in API calls in a frame). Same thing happens when the load is reduced suddenly (i.e. turning the camera away from a heavy model).
After the stall it runs smoothly, until the load is changed suddenly again. It happens constantly, not just when the model is first viewed. It's to do with change in work load (presumably CPU workload in the driver thread). What is more telling is that during the stalls there's a constant stream of 100's of page faults shown in task manager for the application rendering the model. When the stalling stops, the page faults stop.
5 second stalls are not acceptable. This occurs on Quadro cards from the 3500 up to the 5800, on both XP (32 and 64 bit) and Vista/w7, on dual and quad core CPU's, on 2GB memory and 8GB memory systems. It happens everywhere. It also happens in other third party applications rendering the same kind of scenes.

peterfilm
03-10-2010, 09:17 AM
...5 second stalls when the load suddenly increases (i.e. when you turn the camera to face the model...
Check the amount of texture, VBOs, etc. you're using. If you're blowing past the amount that comfortably fits in GPU memory then such stalls are expected, particularly when you turn the camera. Though 5 sec sounds a bit long. Also, you do know you have to prerender to force the textures/etc. onto the card, right?

This may be the dynamic shader recompilation when uniforms change annoyance, but that was mostly a pre-GeForce 8 thing IIRC.

BTW, shouldn't this have been a new thread?

peterfilm
03-10-2010, 09:18 AM
no textures, no VBO's, memory not the issue (one of the cards has 4GB onboard, but doesn't matter anyway as the biggest scene i've tested is only 250MB in size).
The bit about pre-rendering - yes i now about that, but you obviously missed the bit in my post where i said it was consistently stalling for 5 seconds throughout the run, not just the first time.
The only shader uniform being changed is the current modelview matrix (via glLoadMatrix). This is using the fixed function pipeline and using a shader that uses the built-in state (alternating between the two code paths at build-time, not run-time). I've basically traced the problem to glLoadMatrix, but seeing as though that's just a uniform under the hood, I imagine it happens for any 16 float uniform changed at that frequency...although I haven't checked.

You're absolutely right - this perhaps deserves a new thread. Which I have just done now.

peterfilm
03-10-2010, 01:54 PM
difficult to believe nobody else is experiencing this problem. I can reproduce it without any effort whatsoever.



void draw()
{
int batchcount = random(50000);
for (int i=0; i<batchcount; ++i)
{
glPushMatrix();
glTranslatef(-10.0f+random(20.0f), -10.0f+random(20.0f), -10.0f+random(20.0f);
glCallList(dlistBaseID+random(numberOfDisplayLists ));
glPopMatrix();
}
}

The display lists are at the most a couple of hundred triangles. The card is spec'd as capable of 300 million triangles per second and has 4GB of memory. It happens on a wide variety of driver versions too. Haven't rolled back the driver until I find a version where it doesn't happen, because I believe that's the job of NVidia. All I know is it's been introduced within the last year, as these models rendered fine over a year ago.
Page faults insanely, framerate drops to seconds-per-frame, then back up to frames-per-second, then back down again.
Nobody getting the same? I'd be grateful if someone could try to reproduce it.

peterfilm
03-10-2010, 02:05 PM
actually, because i need an answer quite quickly, i'll paste the code to a simple GLUT program that reproduces the problem. If some dear soul could copy/paste it, build it and let me know that it happens to them too, i'd be very grateful. I hate it when people paste big blocks of code onto forums, but i'm at my wits end.


#include <glut/glut.h>
#include <math.h>
#include <stdio.h>
#include <stdlib.h>

template <typename T> inline T rnd(T maxVal) {return T((double(rand())/double(RAND_MAX)) * double(maxVal));}

int g_dlistCount = 4000;
int g_batchCount = 50000;
GLuint* g_dlists = 0;
int g_TimeMs = 0;
float g_TimeDeltaSeconds = 0.0f;
int g_batchTriCount = 0;

static bool sceneInitialise()
{
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glDisable(GL_FOG);
glDisable(GL_CULL_FACE);
glDisable(GL_BLEND);

glEnable(GL_LIGHTING);
glEnable(GL_LIGHT0);
glEnable(GL_COLOR_MATERIAL);

int sides = 10;
int rings = 20;

g_batchTriCount = sides*rings*2; // assuming 2 tris per quad

printf("Creating %i display lists, each containing %i triangles (total:%.2f million triangles).\n",
g_dlistCount, g_batchTriCount, double(g_batchTriCount*g_dlistCount)/1000000.0);

// create some random display lists
g_dlists = new GLuint[g_dlistCount];
for (int i=0; i<g_dlistCount; ++i)
{
float colour[3]={rnd(1.0f),rnd(1.0f),rnd(1.0f)};

g_dlists[i]=glGenLists(1);
glNewList(g_dlists[i], GL_COMPILE);
glColor3fv(colour);
glutSolidTorus(rnd(0.1f), rnd(2.0f), sides, rings);
glEndList();
}

printf("Each frame will randomly call up to %i display lists.\n", g_batchCount);
printf("The maximum number of triangles rendered will be %.2f million.\n", double(g_batchCount*g_batchTriCount)/1000000.0);

return true;
}

static void sceneDraw()
{
// pick a random number of batches to draw this frame
srand(g_TimeMs/16);
rand();
int frameBatchCount = rnd(g_batchCount-1)+1;

// seed to constant, so all batches are the same each frame.
srand(2010);

printf("[%.4f] about to draw %i batches (%.2f million triangles)...", g_TimeDeltaSeconds, frameBatchCount, double(frameBatchCount*g_batchTriCount)/1000000.0);

glMatrixMode(GL_MODELVIEW);
for (int i=0; i<frameBatchCount; ++i)
{
glPushMatrix();
{
// set a random position
glTranslatef(-3.0f+rnd(6.0f), -3.0f+rnd(6.0f), -3.0f+rnd(6.0f));

// make a random unit length axis
float axis[3]={-1.0f+rnd(2.0f), -1.0f+rnd(2.0f), -1.0f+rnd(2.0f)};
float axisLen = sqrtf(axis[0]*axis[0]+axis[1]*axis[1]+axis[2]*axis[2]);
if (axisLen>0.0f) for (int q=0;q<3;++q) axis[q]/=axisLen;

// rotate by random amount around the random axis
glRotatef(rnd(360.0f), axis[0], axis[1], axis[2]);

// draw a random model
glCallList(g_dlists[rnd(g_dlistCount-1)]);
}
glPopMatrix();
}

printf("finished drawing.\n");
}

static void glutDisplay()
{
g_TimeMs = glutGet(GLUT_ELAPSED_TIME);
static int prevMs = g_TimeMs;
g_TimeDeltaSeconds = float(g_TimeMs-prevMs)/1000.0f;
prevMs = g_TimeMs;

glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT);

static float sceneRotation = 0.0f;
const float degreesPerSecond = 100.0f;
sceneRotation += degreesPerSecond * g_TimeDeltaSeconds;

glMatrixMode(GL_MODELVIEW);
glPushMatrix();
{
glTranslatef(0.0f, 0.0f, -20.0f);
glRotatef(sceneRotation, 0.0f, 1.0f, 0.0f);

sceneDraw();
}
glPopMatrix();

glutSwapBuffers();
}

static void glutReshape(int w, int h)
{
glViewport(0, 0, w, h);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(45.0, (GLfloat)w/(GLfloat)h, 0.1, 10000.0);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
}

int main(int argc, char* argv[])
{
glutInitWindowSize(1024, 768);
glutInit(&amp;argc, argv);
glutInitDisplayMode(GLUT_RGBA|GLUT_DEPTH|GLUT_DOUB LE);
glutCreateWindow("StutterReproducer");
glutReshapeFunc(glutReshape);
glutDisplayFunc(glutDisplay);
glutIdleFunc(glutDisplay);
if (sceneInitialise())
glutMainLoop();
return 0;
}

peterfilm
03-10-2010, 02:24 PM
running this test program on a much lowlier geforce 8600M GS (basically on an old laptop) gives fine performance with no page faults per frame. I haven't tested it on a normal desktop geforce, so I'm assuming it's a Quadro-only problem. Anyway, once again, eternal gratitude to anyone who takes the plunge and gives it a bit of a test (on quadro or desktop geforce).

BTW, I'm not saying this test program stutters for 5 seconds, the models i'm trying to render do, but the test program stutters for 1.5 seconds on average (when there's, say, a frame rendered with 13 million triangles followed by a frame rendered with 18 million).

nsdev
03-10-2010, 03:32 PM
Hello,

I ran your test app on a linux box at home.
Here's part of the output:

GL vendor = NVIDIA Corporation
GL renderer = GeForce GTX 260/PCI/SSE2
GL version = 3.2.0 NVIDIA 190.53
Creating 4000 display lists, each containing 400 triangles (total:1.60 million triangles).
Each frame will randomly call up to 50000 display lists.
The maximum number of triangles rendered will be 20.00 million.
[0.0000] about to draw 37006 batches (14.80 million triangles)...finished drawing.
[0.0970] about to draw 10766 batches (4.31 million triangles)...finished drawing.
[0.0180] about to draw 2248 batches (0.90 million triangles)...finished drawing.
[0.0060] about to draw 2248 batches (0.90 million triangles)...finished drawing.
[0.0140] about to draw 23096 batches (9.24 million triangles)...finished drawing.
[0.0510] about to draw 34952 batches (13.98 million triangles)...finished drawing.
[0.0400] about to draw 1430 batches (0.57 million triangles)...finished drawing.
[0.0240] about to draw 17463 batches (6.99 million triangles)...finished drawing.
[0.0210] about to draw 38057 batches (15.22 million triangles)...finished drawing.
[0.0550] about to draw 45584 batches (18.23 million triangles)...finished drawing.
[0.0770] about to draw 24275 batches (9.71 million triangles)...finished drawing.
[0.0470] about to draw 15438 batches (6.18 million triangles)...finished drawing.
[0.0290] about to draw 31432 batches (12.57 million triangles)...finished drawing.
[0.0590] about to draw 39094 batches (15.64 million triangles)...finished drawing.
[0.0680] about to draw 46883 batches (18.75 million triangles)...finished drawing.
[0.0870] about to draw 45948 batches (18.38 million triangles)...finished drawing.
[0.0690] about to draw 3649 batches (1.46 million triangles)...finished drawing.
[0.0240] about to draw 49159 batches (19.66 million triangles)...finished drawing.
[0.0630] about to draw 6433 batches (2.57 million triangles)...finished drawing.
[0.0220] about to draw 22891 batches (9.16 million triangles)...finished drawing.
[0.0320] about to draw 14391 batches (5.76 million triangles)...finished drawing.
[0.0280] about to draw 5674 batches (2.27 million triangles)...finished drawing.
[0.0320] about to draw 46954 batches (18.78 million triangles)...finished drawing.
[0.0560] about to draw 8939 batches (3.58 million triangles)...finished drawing.
[0.0320] about to draw 58 batches (0.02 million triangles)...finished drawing.
[0.0170] about to draw 46086 batches (18.43 million triangles)...finished drawing.
[0.0530] about to draw 32947 batches (13.18 million triangles)...finished drawing.
[0.0720] about to draw 11012 batches (4.40 million triangles)...finished drawing.
[0.0120] about to draw 31803 batches (12.72 million triangles)...finished drawing.
[0.0500] about to draw 43730 batches (17.49 million triangles)...finished drawing.
[0.0730] about to draw 1157 batches (0.46 million triangles)...finished drawing.
[0.0280] about to draw 17587 batches (7.03 million triangles)...finished drawing.
[0.0210] about to draw 38303 batches (15.32 million triangles)...finished drawing.
[0.0550] about to draw 20929 batches (8.37 million triangles)...finished drawing.
[0.0350] about to draw 12180 batches (4.87 million triangles)...finished drawing.
[0.0310] about to draw 3454 batches (1.38 million triangles)...finished drawing.
[0.0290] about to draw 44873 batches (17.95 million triangles)...finished drawing.
[0.0560] about to draw 6614 batches (2.65 million triangles)...finished drawing.
[0.0240] about to draw 23237 batches (9.29 million triangles)...finished drawing.
[0.0550] about to draw 10135 batches (4.05 million triangles)...finished drawing.
[0.0110] about to draw 5401 batches (2.16 million triangles)...finished drawing.
[0.0060] about to draw 5401 batches (2.16 million triangles)...finished drawing.
[0.0140] about to draw 26286 batches (10.51 million triangles)...finished drawing.
[0.0320] about to draw 17611 batches (7.04 million triangles)...finished drawing.
[0.0380] about to draw 4627 batches (1.85 million triangles)...finished drawing.
[0.0290] about to draw 202 batches (0.08 million triangles)...finished drawing.
[0.0110] about to draw 21181 batches (8.47 million triangles)...finished drawing.
[0.0240] about to draw 12236 batches (4.89 million triangles)...finished drawing.
[0.0240] about to draw 33080 batches (13.23 million triangles)...finished drawing.
[0.0520] about to draw 19971 batches (7.99 million triangles)...finished drawing.
[0.0340] about to draw 36136 batches (14.45 million triangles)...finished drawing.
[0.0700] about to draw 39228 batches (15.69 million triangles)...finished drawing.
[0.0680] about to draw 47020 batches (18.81 million triangles)...finished drawing.
[0.0990] about to draw 45945 batches (18.38 million triangles)...finished drawing.
[0.0620] about to draw 3644 batches (1.46 million triangles)...finished drawing.
[0.0330] about to draw 19971 batches (7.99 million triangles)...finished drawing.
[0.0230] about to draw 11135 batches (4.45 million triangles)...finished drawing.
[0.0320] about to draw 2474 batches (0.99 million triangles)...finished drawing.
[0.0280] about to draw 23118 batches (9.25 million triangles)...finished drawing.
[0.0260] about to draw 14468 batches (5.79 million triangles)...finished drawing.
[0.0310] about to draw 30951 batches (12.38 million triangles)...finished drawing.
[0.0540] about to draw 17757 batches (7.10 million triangles)...finished drawing.
[0.0290] about to draw 9083 batches (3.63 million triangles)...finished drawing.
[0.0350] about to draw 258 batches (0.10 million triangles)...finished drawing.
[0.0230] about to draw 41828 batches (16.73 million triangles)...finished drawing.
[0.0480] about to draw 28816 batches (11.53 million triangles)...finished drawing.
[0.0640] about to draw 36119 batches (14.45 million triangles)...finished drawing.
[0.0390] about to draw 27404 batches (10.96 million triangles)...finished drawing.
[0.0360] about to draw 18796 batches (7.52 million triangles)...finished drawing.
[0.0380] about to draw 5784 batches (2.31 million triangles)...finished drawing.
[0.0380] about to draw 22169 batches (8.87 million triangles)...finished drawing.
[0.0240] about to draw 38360 batches (15.34 million triangles)...finished drawing.
[0.0610] about to draw 25524 batches (10.21 million triangles)...finished drawing.
[0.0480] about to draw 12310 batches (4.92 million triangles)...finished drawing.
[0.0300] about to draw 28715 batches (11.49 million triangles)...finished drawing.
[0.0480] about to draw 15486 batches (6.19 million triangles)...finished drawing.
[0.0260] about to draw 31869 batches (12.75 million triangles)...finished drawing.
[0.0570] about to draw 14529 batches (5.81 million triangles)...finished drawing.
[0.0590] about to draw 1395 batches (0.56 million triangles)...finished drawing.
[0.0010] about to draw 1395 batches (0.56 million triangles)...finished drawing.
[0.0040] about to draw 21895 batches (8.76 million triangles)...finished drawing.
[0.0310] about to draw 42748 batches (17.10 million triangles)...finished drawing.
[0.0690] about to draw 21132 batches (8.45 million triangles)...finished drawing.
[0.0350] about to draw 37446 batches (14.98 million triangles)...finished drawing.
[0.0730] about to draw 40447 batches (16.18 million triangles)...finished drawing.
[0.0710] about to draw 23509 batches (9.40 million triangles)...finished drawing.
[0.0460] about to draw 10272 batches (4.11 million triangles)...finished drawing.
[0.0340] about to draw 1538 batches (0.62 million triangles)...finished drawing.
[0.0210] about to draw 47054 batches (18.82 million triangles)...finished drawing.
[0.0530] about to draw 29804 batches (11.92 million triangles)...finished drawing.
[0.0670] about to draw 12053 batches (4.82 million triangles)...finished drawing.
[0.0140] about to draw 8064 batches (3.23 million triangles)...finished drawing.
[0.0190] about to draw 3813 batches (1.53 million triangles)...finished drawing.
[0.0220] about to draw 24503 batches (9.80 million triangles)...finished drawing.
[0.0280] about to draw 15546 batches (6.22 million triangles)...finished drawing.
[0.0340] about to draw 31981 batches (12.79 million triangles)...finished drawing.
[0.0560] about to draw 39840 batches (15.94 million triangles)...finished drawing.
[0.0670] about to draw 22295 batches (8.92 million triangles)...finished drawing.
[0.0390] about to draw 38684 batches (15.47 million triangles)...finished drawing.
[0.0720] about to draw 41821 batches (16.73 million triangles)...finished drawing.
[0.0720] about to draw 49405 batches (19.76 million triangles)...finished drawing.
[0.0970] about to draw 48594 batches (19.44 million triangles)...finished drawing.
[0.0690] about to draw 1578 batches (0.63 million triangles)...finished drawing.
[0.0300] about to draw 47294 batches (18.92 million triangles)...finished drawing.
[0.0550] about to draw 29568 batches (11.83 million triangles)...finished drawing.
[0.0520] about to draw 41858 batches (16.74 million triangles)...finished drawing.
[0.0710] about to draw 20212 batches (8.08 million triangles)...finished drawing.
[0.0410] about to draw 36419 batches (14.57 million triangles)...finished drawing.
[0.0620] about to draw 19348 batches (7.74 million triangles)...finished drawing.
[0.0390] about to draw 35451 batches (14.18 million triangles)...finished drawing.
[0.0790] about to draw 13591 batches (5.44 million triangles)...finished drawing.
[0.0150] about to draw 9334 batches (3.73 million triangles)...finished drawing.
[0.0130] about to draw 4676 batches (1.87 million triangles)...finished drawing.
[0.0200] about to draw 599 batches (0.24 million triangles)...finished drawing.
[0.0150] about to draw 46102 batches (18.44 million triangles)...finished drawing.
[0.0570] about to draw 29249 batches (11.70 million triangles)...finished drawing.
[0.0540] about to draw 15732 batches (6.29 million triangles)...finished drawing.
[0.0340] about to draw 7043 batches (2.82 million triangles)...finished drawing.
[0.0320] about to draw 23317 batches (9.33 million triangles)...finished drawing.
[0.0310] about to draw 14873 batches (5.95 million triangles)...finished drawing.
[0.0250] about to draw 5914 batches (2.37 million triangles)...finished drawing.
[0.0360] about to draw 22264 batches (8.91 million triangles)...finished drawing.
[0.0280] about to draw 38723 batches (15.49 million triangles)...finished drawing.
[0.0650] about to draw 46081 batches (18.43 million triangles)...finished drawing.
[0.0860] about to draw 24596 batches (9.84 million triangles)...finished drawing.
[0.0300] about to draw 15817 batches (6.33 million triangles)...finished drawing.
[0.0350] about to draw 32136 batches (12.85 million triangles)...finished drawing.
[0.0580] about to draw 39623 batches (15.85 million triangles)...finished drawing.
[0.0690] about to draw 47310 batches (18.92 million triangles)...finished drawing.
[0.0800] about to draw 613 batches (0.25 million triangles)...finished drawing.
[0.0240] about to draw 16938 batches (6.78 million triangles)...finished drawing.
[0.0190] about to draw 37495 batches (15.00 million triangles)...finished drawing.
[0.0600] about to draw 45114 batches (18.05 million triangles)...finished drawing.
[0.0960] about to draw 43916 batches (17.57 million triangles)...finished drawing.
[0.0580] about to draw 6060 batches (2.42 million triangles)...finished drawing.
[0.0330] about to draw 47408 batches (18.96 million triangles)...finished drawing.
[0.0630] about to draw 5198 batches (2.08 million triangles)...finished drawing.
[0.0290] about to draw 21467 batches (8.59 million triangles)...finished drawing.
[0.0240] about to draw 37565 batches (15.03 million triangles)...finished drawing.
[0.0630] about to draw 20275 batches (8.11 million triangles)...finished drawing.
[0.0350] about to draw 36554 batches (14.62 million triangles)...finished drawing.
[0.0700] about to draw 19310 batches (7.72 million triangles)...finished drawing.
[0.0390] about to draw 5960 batches (2.38 million triangles)...finished drawing.
[0.0250] about to draw 26696 batches (10.68 million triangles)...finished drawing.
[0.0550] about to draw 34485 batches (13.79 million triangles)...finished drawing.
[0.0410] about to draw 25914 batches (10.37 million triangles)...finished drawing.
[0.0490] about to draw 12734 batches (5.09 million triangles)...finished drawing.
[0.0340] about to draw 4303 batches (1.72 million triangles)...finished drawing.
[0.0290] about to draw 20299 batches (8.12 million triangles)...finished drawing.
[0.0240] about to draw 36577 batches (14.63 million triangles)...finished drawing.
[0.0630] about to draw 19315 batches (7.73 million triangles)...finished drawing.
[0.0330] about to draw 35586 batches (14.23 million triangles)...finished drawing.
[0.0680] about to draw 43137 batches (17.25 million triangles)...finished drawing.
[0.0730] about to draw 21303 batches (8.52 million triangles)...finished drawing.
[0.0420] about to draw 12929 batches (5.17 million triangles)...finished drawing.
[0.0510] about to draw 24716 batches (9.89 million triangles)...finished drawing.
[0.0320] about to draw 16088 batches (6.44 million triangles)...finished drawing.
[0.0230] about to draw 7471 batches (2.99 million triangles)...finished drawing.
[0.0340] about to draw 23610 batches (9.44 million triangles)...finished drawing.
[0.0270] about to draw 39967 batches (15.99 million triangles)...finished drawing.
[0.0660] about to draw 22739 batches (9.10 million triangles)...finished drawing.
[0.0440] about to draw 38750 batches (15.50 million triangles)...finished drawing.
[0.0680] about to draw 41924 batches (16.77 million triangles)...finished drawing.
[0.0720] about to draw 24573 batches (9.83 million triangles)...finished drawing.
[0.0490] about to draw 36451 batches (14.58 million triangles)...finished drawing.
[0.0830] about to draw 40046 batches (16.02 million triangles)...finished drawing.
[0.0520] about to draw 22804 batches (9.12 million triangles)...finished drawing.
[0.0440] about to draw 13951 batches (5.58 million triangles)...finished drawing.
[0.0300] about to draw 30184 batches (12.07 million triangles)...finished drawing.
[0.0530] about to draw 37909 batches (15.16 million triangles)...finished drawing.
[0.0670] about to draw 45202 batches (18.08 million triangles)...finished drawing.
[0.0750] about to draw 48822 batches (19.53 million triangles)...finished drawing.
[0.0820] about to draw 2063 batches (0.83 million triangles)...finished drawing.
[0.0240] about to draw 47846 batches (19.14 million triangles)...finished drawing.
[0.0790] about to draw 25761 batches (10.30 million triangles)...finished drawing.
[0.0290] about to draw 42205 batches (16.88 million triangles)...finished drawing.
[0.0750] about to draw 45539 batches (18.22 million triangles)...finished drawing.
[0.0770] about to draw 28050 batches (11.22 million triangles)...finished drawing.
[0.0530] about to draw 10668 batches (4.27 million triangles)...finished drawing.
[0.0330] about to draw 27117 batches (10.85 million triangles)...finished drawing.
[0.0420] about to draw 43391 batches (17.36 million triangles)...finished drawing.
[0.0750] about to draw 21424 batches (8.57 million triangles)...finished drawing.
[0.0420] about to draw 8506 batches (3.40 million triangles)...finished drawing.
[0.0310] about to draw 24730 batches (9.89 million triangles)...finished drawing.
[0.0440] about to draw 16038 batches (6.42 million triangles)...finished drawing.
[0.0180] about to draw 32450 batches (12.98 million triangles)...finished drawing.
[0.0510] about to draw 19301 batches (7.72 million triangles)...finished drawing.
[0.0380] about to draw 35621 batches (14.25 million triangles)...finished drawing.
[0.0650] about to draw 18252 batches (7.30 million triangles)...finished drawing.
[0.0360] about to draw 34800 batches (13.92 million triangles)...finished drawing.
[0.0630] about to draw 17328 batches (6.93 million triangles)...finished drawing.
[0.0290] about to draw 33655 batches (13.46 million triangles)...finished drawing.
[0.0660] about to draw 16356 batches (6.54 million triangles)...finished drawing.
[0.0280] about to draw 32813 batches (13.13 million triangles)...finished drawing.
[0.0570] about to draw 15043 batches (6.02 million triangles)...finished drawing.
[0.0500] about to draw 2138 batches (0.86 million triangles)...finished drawing.
[0.0060] about to draw 2138 batches (0.86 million triangles)...finished drawing.
[0.0040] about to draw 2138 batches (0.86 million triangles)...finished drawing.
[0.0070] about to draw 22791 batches (9.12 million triangles)...finished drawing.
[0.0250] about to draw 43486 batches (17.39 million triangles)...finished drawing.
[0.0780] about to draw 46394 batches (18.56 million triangles)...finished drawing.
[0.0780] about to draw 49816 batches (19.93 million triangles)...finished drawing.
[0.0830] about to draw 3416 batches (1.37 million triangles)...finished drawing.
[0.0310] about to draw 44518 batches (17.81 million triangles)...finished drawing.
[0.0500] about to draw 31578 batches (12.63 million triangles)...finished drawing.
[0.0580] about to draw 39007 batches (15.60 million triangles)...finished drawing.
[0.0850] about to draw 17228 batches (6.89 million triangles)...finished drawing.
[0.0190] about to draw 12940 batches (5.18 million triangles)...finished drawing.
[0.0200] about to draw 29413 batches (11.77 million triangles)...finished drawing.
[0.0500] about to draw 41337 batches (16.53 million triangles)...finished drawing.
[0.0720] about to draw 48900 batches (19.56 million triangles)...finished drawing.
[0.0800] about to draw 2345 batches (0.94 million triangles)...finished drawing.
[0.0290] about to draw 43513 batches (17.41 million triangles)...finished drawing.
[0.0490] about to draw 5342 batches (2.14 million triangles)...finished drawing.
[0.0290] about to draw 21773 batches (8.71 million triangles)...finished drawing.
[0.0280] about to draw 13102 batches (5.24 million triangles)...finished drawing.
[0.0330] about to draw 4393 batches (1.76 million triangles)...finished drawing.
[0.0290] about to draw 45680 batches (18.27 million triangles)...finished drawing.
[0.0730] about to draw 28468 batches (11.39 million triangles)...finished drawing.
[0.0320] about to draw 19420 batches (7.77 million triangles)...finished drawing.
[0.0380] about to draw 6524 batches (2.61 million triangles)...finished drawing.
[0.0370] about to draw 47710 batches (19.08 million triangles)...finished drawing.
[0.0540] about to draw 9503 batches (3.80 million triangles)...finished drawing.
[0.0280] about to draw 26170 batches (10.47 million triangles)...finished drawing.
[0.0460] about to draw 13093 batches (5.24 million triangles)...finished drawing.

peterfilm
03-10-2010, 03:49 PM
mmm, no spikes at all for you (the first float on the frame print is the number of seconds the previous frame took to render...I get spikes of 1.5 to 2.0 seconds when the load is suddenly increased). God only knows if its linux or the geforce that's not exhibiting the problem. I'm running on various versions of Windows. Thanks, useful feedback.

peterfilm
03-10-2010, 03:57 PM
by the way, my nvidia devrel login isn't working any more, so my options are limited. I'm just hoping someone from nvidia sees this post and pulls their finger out.

ZbuffeR
03-10-2010, 04:43 PM
No big stuttering either :

Vista 64bits sp2, forceware 191.07

(I added to your code :
printf("GL_VERSION: %s\n",glGetString(GL_VERSION) );
printf("GL_VENDOR: %s\n",glGetString(GL_VENDOR) );
printf("GL_RENDERER: %s\n",glGetString(GL_RENDERER) );
so it the output is more informative)

GL_VERSION: 3.2.0
GL_VENDOR: NVIDIA Corporation
GL_RENDERER: GeForce GTX 275/PCI/SSE2
Creating 4000 display lists, each containing 400 triangles (total:1.60 million triangles).
Each frame will randomly call up to 50000 display lists.
The maximum number of triangles rendered will be 20.00 million.
[0.0000] about to draw 37402 batches (14.96 million triangles)...finished drawing.
[0.0130] about to draw 3802 batches (1.52 million triangles)...finished drawing.
[0.1800] about to draw 34211 batches (13.68 million triangles)...finished drawing.
[0.0100] about to draw 611 batches (0.24 million triangles)...finished drawing.
[0.0680] about to draw 16215 batches (6.49 million triangles)...finished drawing.
[0.0050] about to draw 16215 batches (6.49 million triangles)...finished drawing.
[0.0220] about to draw 49017 batches (19.61 million triangles)...finished drawing.
[0.0340] about to draw 31817 batches (12.73 million triangles)...finished drawing.
[0.0660] about to draw 47421 batches (18.97 million triangles)...finished drawing.
[0.0500] about to draw 46623 batches (18.65 million triangles)...finished drawing.
[0.0670] about to draw 12226 batches (4.89 million triangles)...finished drawing.
[0.0670] about to draw 44230 batches (17.69 million triangles)...finished drawing.
[0.0170] about to draw 10630 batches (4.25 million triangles)...finished drawing.
[0.0660] about to draw 26234 batches (10.49 million triangles)...finished drawing.
[0.0170] about to draw 42634 batches (17.05 million triangles)...finished drawing.
[0.0350] about to draw 25436 batches (10.17 million triangles)...finished drawing.
[0.0650] about to draw 41039 batches (16.42 million triangles)...finished drawing.
[0.0340] about to draw 23841 batches (9.54 million triangles)...finished drawing.
[0.0540] about to draw 39443 batches (15.78 million triangles)...finished drawing.
[0.0320] about to draw 22245 batches (8.90 million triangles)...finished drawing.
[0.0520] about to draw 21447 batches (8.58 million triangles)...finished drawing.
[0.0450] about to draw 20649 batches (8.26 million triangles)...finished drawing.
[0.0330] about to draw 3451 batches (1.38 million triangles)...finished drawing.
[0.0340] about to draw 36253 batches (14.50 million triangles)...finished drawing.
[0.0110] about to draw 36253 batches (14.50 million triangles)...finished drawing.
[0.0440] about to draw 35455 batches (14.18 million triangles)...finished drawing.
[0.0470] about to draw 34658 batches (13.86 million triangles)...finished drawing.
[0.0480] about to draw 33860 batches (13.54 million triangles)...finished drawing.
[0.0500] about to draw 33062 batches (13.22 million triangles)...finished drawing.
[0.0500] about to draw 32264 batches (12.91 million triangles)...finished drawing.
[0.0500] about to draw 47868 batches (19.15 million triangles)...finished drawing.
[0.0500] about to draw 47070 batches (18.83 million triangles)...finished drawing.
[0.0670] about to draw 12673 batches (5.07 million triangles)...finished drawing.
[0.0670] about to draw 28275 batches (11.31 million triangles)...finished drawing.
[0.0170] about to draw 44677 batches (17.87 million triangles)...finished drawing.
[0.0380] about to draw 27477 batches (10.99 million triangles)...finished drawing.
[0.0620] about to draw 43081 batches (17.23 million triangles)...finished drawing.
[0.0360] about to draw 42283 batches (16.91 million triangles)...finished drawing.
[0.0640] about to draw 7886 batches (3.15 million triangles)...finished drawing.
[0.0550] about to draw 7088 batches (2.84 million triangles)...finished drawing.

So, Quadro-only bug ?

peterfilm
03-11-2010, 01:40 AM
thanks so very much for the effort.
it does seem to confirm this is a quadro only bug, yes.
the quadro probably has a different display list compiler (this doesn't happen with VBO's, just with display lists).

barthold
03-11-2010, 10:40 AM
Peterfilm,

We'll take a look. What driver version are you seeing this on?

Thanks,
Barthold
(with my NVIDIA hat on)

peterfilm
03-11-2010, 11:05 AM
oh thank heavens, thank you Barthold!
the driver i'm currently on at work is 191.87 (quadro 5800).
but as I say, it happens on lots of earlier drivers. Lots of quadro people around the office have different levels of complacency about updating drivers, so it's had a fair old test on the last 6 months worth. Always happens, without exception.
It also happens on a certain (well known in the industry) engineering package. That's why I'm surprised it hasn't been fixed yet (I've been hoping it would go away since at least november).

(sorry for the "pull their finger out" quip, it's not nvidia's fault if the bug hasn't been reported before...it's just my turn of phrase coming out all wrong).

ZbuffeR
03-11-2010, 01:43 PM
It also happens on a certain (well known in the industry) engineering package
Which one ?

peterfilm
03-11-2010, 01:59 PM
i'd rather not say. We have a business relationship with them, so it wouldn't be cricket to highlight it's having problems...even if those problems are not its fault. Might sound overly cautious, but I'm becoming that way in my old age.

peterfilm
03-13-2010, 06:05 AM
<bump>
I hate it when people do that too....but it's probably more than obvious just how desperate I am for some kind of answer (and promise). Got customers waiting for a solution. When you're in a CAVE, a 5 second halt when you turn your head is quite disturbing. The fourth wall is well and truly smashed. I've given them the VBO version for the moment, but it runs about 10% slower than the display list version (10% amounts to a lot when you're viewing models of this kind of size).
(I'm also desperately trying to knock together a bindless graphics version to try to reduce the 10%....as if I haven't got enough other things to be doing!!)

barthold
03-16-2010, 08:20 AM
You have a PM.

Barthold

cscheid
05-11-2010, 03:05 PM
Sorry to revive an old thread, but I am also seeing something very similar, albeit on a smaller scale. I am running a GTX260 on x86_64 Ubuntu 9.04, 12GB RAM, 2*quad core Xeons. glxinfo gives



...
OpenGL renderer string: GeForce GTX 260/PCI/SSE2
OpenGL version string: 3.2.0 NVIDIA 190.42
...


My original application sends large batches of client-side data via glVertexPointer (the data does not fit in video memory - think hundreds of millions of points), and I was seeing similar stutters similar to what peterfilm described. I ran his code and the output looks like this:



[0.0170] about to draw 32730 batches (13.09 million triangles)...finished drawing.
[0.0440] about to draw 24275 batches (9.71 million triangles)...finished drawing.
[0.2160] about to draw 13248 batches (5.30 million triangles)...finished drawing.
[0.0170] about to draw 8923 batches (3.57 million triangles)...finished drawing.
...
[0.0400] about to draw 46086 batches (18.43 million triangles)...finished drawing.
[0.2500] about to draw 1157 batches (0.46 million triangles)...finished drawing.
[0.0020] about to draw 1157 batches (0.46 million triangles)...finished drawing.
[0.0010] about to draw 1157 batches (0.46 million triangles)...finished drawing.


Notice that there are relatively large hiccups happening on relatively small batches. On my original (GLUT) application, I know that the freezes happen on glutSwapBuffer calls, but I haven't checked whether this is the case for peterfilm's application as well.

Is there by now a known fix I can try? This is driving me up the wall...

Thank you so much.

cscheid
05-11-2010, 03:23 PM
This is just to add that I have just upgraded to 195.36.24 and I still see similar hiccups.

I take that back - peterfilm's code does not cause hiccups on 195.36.24, but my application still exhibits problems, which tells me that they are separate issues. I have posted the full description of my hiccuping issue in a new topic (http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&amp;Number=277541#Post2775 41), in case anyone cares. (Took me a while since it was a fairly delicate bug to trigger)

cbell
06-16-2010, 10:10 AM
Have just found this thread and am so grateful because I thought it was just me!!

I have 4 core (2.93GHz Xeon) machine with a Quadro FX3800 and 24GBytes of RAM, bootable to Windows 7, Windows XP and CentOS 5.3 - all 64 bit.

I too am seeing this "stutter". On Windows 7 it is OK-ish with the 191.87 driver, but really quite severe with 197.03 and 197.90. On Linux it is much worse. I haven't really done much on XP-64 other than to establish that Bill Gates' stupid "Aero Desktop" slows down graphics by 25%.

It's a large engineering package (although not, I suspect, the one mentioned below) which is animating typically 1.5 million mixed quads and trias with some line overlay. A typical animation might contain 40 frames or so, and every frame has different vertex coordinates but much the same facet topology.

It will trundle along quite happily at - say - 40mS/frame for maybe 50 frames, then just stop for perhaps a second, and then carry on again.

Stalling is seemingly random, but when it happens I notice that my working set size will increase from (say) 1.3GB to 1.5GB, page faults also leap. Once the stall is over the working set size drops back again to what it was before.

Annoyingly on Windows it won't do this under debug, only with the release build, so it is hard to tell what it is doing (or not doing!) when it stalls.

However under Linux I have managed to stop it in the debugger when it is doing this, and it is deep inside the NVidia stuff (the calling stack includes several layers of routines called nvgl + some numbers) which is then calling malloc(). So that explains the rise in memory use + the page faults.


I see this behaviour both if I render in immediate mode, and if I use vertex arrays, and if I use vertex buffer objects for vertex coordinates and/or array element indices.

It is worst when I combine vertex arrays with a GLSL fragment shader to do the lighting calcs for me, since this means that I don't have to send normals so the rate at which the facets arrive for rendering is faster. (Shaders rock - using them increases my frame rate by at least x3)

It feels as if I am managing to throw data at the driver faster than it can handle it so that it has to pause, store it temporarily somewhere (hence the malloc), and then carry on once it has sorted itself out. Certainly when it is running smoothly on Windows it is typically using only 80% of a cpu (rendering is not currently threaded), suggesting that it is bound by bandwidth of data transfer to the card.

The machine is not doing anything else, and with 24GB RAM available it is nowhere near running out of memory. The application is lightly threaded, but the rendering is not, so it is really just running on one core and the system has plenty of spare capacity.

I've tried sticking an ATI FirePRO card into the machine and running identically the same executable and I don't see this problem (although ATI pose plenty of other problems ...) so I don't think it's anything I'm doing.

We also build on an assorted collection of other hardware and operating systems, none of which show this behaviour, so I do honestly think it is something in the recent Quadro line and/or their drivers.

Any suggestions would be very welcome since I'm tearing my hair out!

Thanks in advance.