FBO usage : glRenderBufferStorageExt OUT_OF_MEMORY

Hello everyone,

Since two days i’m really in trouble with my offscreen rendering. I work on a FEM software with 3D OpenGL visualisation of structure, and i need a good quality screenshot output. I also need to generate numerous offscreen rendering to create AVI.

I’ve use a traditional method of FBO with a texture and a renderbuffer.

My GC is a Quadro 600 with 1024MB Ram, and my output screenshot is 1280*1024.

Some day ago, i was generating a bunch of bitmap (+/- 500 * 3.2MB each).

And after more or less 60 files i get an INVALID_FRAMEBUFFER_OPERATION_EXT on my glCheckFramebufferStatusEXT() !!

Damn ! What the hell is going on !

So i’ve search line by line in my code with the help of glGetError() and found that the renderbuffer allocation give me an OUT_OF_MEMORY code !

glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT, GL_RGBA8,
hsize_export,vsize_export);

(With H = 1280, V=1024)

I was very surprise because after each off screen rendering i delete my FBO, my RenderBuffer and my Texture.

Pushing the debug ahead i’ve create a simple loop like this :

for( int i =0; i < 500; i++ )
{
GLuint rboId;
glGenRenderbuffersEXT(1, &rboId);
glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, rboId);
glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT,
GL_RGBA8,
hsize_export,vsize_export);
glFinish();
int last_error = glGetError();
if(last_error==1285){AfxMessageBox(“OUT_OF_MEMORY”);}

    glDeleteRenderbuffersEXT(1, &rboId);

}

So basicaly i bind a renderbuffer, i allocate it and i delete it and this 500 time. And then i still get the OUT_OF_MEMORY ERROR. I don’t understand why after +/- 60 cycle of allocate/deallocate i dont have enough memory for my render buffer allocation.

I’m very disappointing with this buf, i’ve check the renderbuffer capacity of my gcard and it’s fine, i go the right drivers from NVIDIA, i’ve read a lot of documentation about INVALID_FRAMEBUFFER_OPERATION_EXT and OUT_OF_MEMORY error but i dont find anything to explain this kind of behaviour.

If you got anything about this problem, i’ll be very pleased to heard it. Sorry for my french accent and grammar

Regards,

Tyrahell

Please use [ code]/[ /code] (without space after ‘[’) around code snippets to improve readability.

but i dont find anything to explain this kind of behaviour.

I’m not sure what sort of explanation you are hoping for, it seems to me you already diagnosed the problem: renderbuffer allocation (apparently and for no good reason) fragments GPU memory and at some point there is no free contiguous chunk available to satisfy your allocation request.
As for why that happens, that is something probably only a driver developer could answer. It may be a bug in the driver or just that nobody bothered to implement a better allocation scheme.
IIRC the general recommendation is to allocate large buffers early in the application and keep them around for the applications lifetime. Arguably this is easier to do for some applications than others
For your application: can you allocate the offscreen render buffers on first use and keep them around without deleting them? You could even code something that keeps them until you get an OUT_OF_MEMORY elsewhere, delete the offscreen buffers then and retry the failed operation.
Might also be worth filing a bug report with Nvidia, including your small reproducer.

Hi Carsten,

As you said “renderbuffer allocation (apparently and for no good reason) fragments GPU memory and at some point there is no free contiguous chunk available to satisfy your allocation request”

Today i’ll try to work on a single renderbuffer allocation at first use for my offscreen rendering and i hope it will solve my problem.

By the way, where can i report this bug to Nvidia ?

Regards,

Tyrahell

Well well well, after another day working on my off screen rendering im stuck again !

I’ve rewrite my FBO allocation to allocate texturebuffer, renderbuffer and FBObuffer once and it seems to work (means no more OUT_OF_MEMORY error).

The problem is that glReadPixels fail at reading my renderbuffer, wich give me result like that :

1st picture out (everything’s fine) :

196th picture out (look buggy as if a part of the first picture was stuck in the render buffer).

I’ve investigate all day about cleaning buffer, usage of GlClear, trying different format to read my renderbuffer and nothing match…

void CGLView::RenderSceneToFile()
{

// Recherche de la definition max supporte par OpenGL
GLint  var;
glGetIntegerv(GL_MAX_RENDERBUFFER_SIZE_EXT, &var);

if ( var > 4096 )
{
	var = 4096;
}


if( hsize_export > var || vsize_export > var )
{
	CString MyMessage = "Definition maximale autorisée : ";
	MyMessage.Format("%s%i",MyMessage,var);
	AfxMessageBox(MyMessage);

	hsize_export = var;
	vsize_export = var;

	RenderFailed = 1;
}
else
{


	if( AllocateFBO == 0 )
	{
		// create a texture object
		glGenTextures(1, &textureId);
		glBindTexture(GL_TEXTURE_2D, textureId);
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR); //GL_LINEAR
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
		glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP, GL_TRUE); // automatic mipmap
		glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8,  hsize_export, vsize_export, 0,
					 GL_RGBA, GL_UNSIGNED_BYTE, 0);
		glBindTexture(GL_TEXTURE_2D, 0);


		// create a renderbuffer object to store depth info
		glGenRenderbuffersEXT(1, &rboId);
		glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, rboId);
		glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT, GL_DEPTH_COMPONENT,hsize_export,vsize_export); 
		glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, 0);

		// create a framebuffer object
		glGenFramebuffersEXT(1, &fboId);
		glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fboId);
		// attach the texture to FBO color attachment point
		glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT,
								  GL_TEXTURE_2D, textureId, 0);

		// attach the renderbuffer to depth attachment point
		glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT,
									 GL_RENDERBUFFER_EXT, rboId);

		AllocateFBO = 1;
	}

	// check FBO status
	GLenum status = glCheckFramebufferStatusEXT( GL_FRAMEBUFFER_EXT );
		

	if( status != GL_FRAMEBUFFER_COMPLETE_EXT )
	{
		VerboseFBO(status);
		RenderFailed = 1;
	}
	else
	{
		float save_m_height = m_height;
		float save_m_width = m_width;
		m_height =  vsize_export;                     
		m_width = hsize_export;   
	 
		float MyRed   = (float)bg_red_export   / (float)255;
		float MyGreen = (float)bg_green_export / (float)255;
		float MyBlue  = (float)bg_blue_export  / (float)255;

		glClearColor( MyRed, MyGreen, MyBlue, 0.5f );


		RenderSceneToOffScreen();

		RenderFBOToFile();
	
		m_height = save_m_height;                     
		m_width = save_m_width;
		glClearColor(m_color[BACKGROUND][0], m_color[BACKGROUND][1], m_color[BACKGROUND][2], 0.5f);
	}

		// switch back to window-system-provided framebuffer
		glBindFramebufferEXT( GL_FRAMEBUFFER_EXT, 0 );
		
}

void CGLView::RenderFBOToFile()
{

	glReadPixels( 0, 0, hsize_export  - 1,  vsize_export - 1, GL_BGR, GL_UNSIGNED_BYTE, ReadBufferGL );

	FILE *file = NULL;	    
	fopen_s( &file, path_export, "wb" );

	// Create & configure Bitmap and File info headers
	BITMAPFILEHEADER bitmapFileHeader;
	BITMAPINFOHEADER bitmapInfoHeader;

	bitmapFileHeader.bfType			= 0x4D42; //"BM" pour BMP
	bitmapFileHeader.bfSize			= vsize_export * hsize_export * 3;
	bitmapFileHeader.bfReserved1	= 0;
	bitmapFileHeader.bfReserved2	= 0;
	bitmapFileHeader.bfOffBits		= sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);
 
	bitmapInfoHeader.biSize			= sizeof(BITMAPINFOHEADER);
	bitmapInfoHeader.biWidth		= hsize_export - 1;
	bitmapInfoHeader.biHeight		= vsize_export - 1;
	bitmapInfoHeader.biPlanes		= 1;
	bitmapInfoHeader.biBitCount		= 24;
	bitmapInfoHeader.biCompression	= BI_RGB;
	bitmapInfoHeader.biSizeImage	= 0;
	bitmapInfoHeader.biXPelsPerMeter= 0; // ?
	bitmapInfoHeader.biYPelsPerMeter= 0; // ?
	bitmapInfoHeader.biClrUsed		= 0;
	bitmapInfoHeader.biClrImportant = 0;

    fwrite(&bitmapFileHeader, sizeof(BITMAPFILEHEADER), 1, file);
	fwrite(&bitmapInfoHeader, sizeof(BITMAPINFOHEADER), 1, file);
	fwrite(ReadBufferGL, vsize_export*hsize_export*3, 1, file);
	fclose(file);

}

If you got any clue about what is failing/missing, i’ll be very pleased to heard about it!

Regards,

Tyrahell

NB : Global byte ReadBufferGL = (byte) malloc(409640963);

Hmm, not sure. Do you perhaps have the wrong glViewport/glScissor settings from rendering to the (presumable smaller) application frame buffer still active?

glReadPixels( 0, 0, hsize_export  - 1,  vsize_export - 1, ...)

arguments 3 and 4 to glReadPixels are width and height, not top right corner of a rectangle, so the -1 looks wrong at first glance.

Hi Carsten !

I’ve modify argument 3 and 4 of glReadPixel but it change nothing.

Since 4 hour i’m working about your glviewport() thought.

I have supposed that if i do something wrong between my screen and off screen rendering and viewport, if i maximize my application on my dual screen i should have a full render in my color buffer and BASINGA it’s true, my color render buffer seems relative to the size of the application windows…

Well, the problem is that i can’t explain why, please can someone check my code if you find someting relevant inside ?

void CGLView::RenderSceneToFile()
{

// Find Max definition of GC
GLint  var;
glGetIntegerv(GL_MAX_RENDERBUFFER_SIZE_EXT, &var);

if ( var > 4096 )
{
	var = 4096;
}


if( hsize_export > var || vsize_export > var )
{
	CString MyMessage = "Definition maximale autorisée : ";
	MyMessage.Format("%s%i",MyMessage,var);
	AfxMessageBox(MyMessage);

	hsize_export = var;
	vsize_export = var;

	RenderFailed = 1;
}
else
{

	if( AllocateFBO == 0 )
	{

		// create a texture object to store color info
		glGenTextures(1, &textureId);
		glBindTexture(GL_TEXTURE_2D, textureId);
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR); //GL_LINEAR
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
		glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP, GL_TRUE); // automatic mipmap
		glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8,  hsize_export, vsize_export, 0,
					 GL_RGBA, GL_UNSIGNED_BYTE, 0);


		// create a renderbuffer object to store depth info
		glGenRenderbuffersEXT(1, &rboId);
		glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, rboId);
		glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT, GL_DEPTH_COMPONENT,hsize_export,vsize_export);
		
		// create a framebuffer object
		glGenFramebuffersEXT(1, &fboId);
		glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fboId);
		
		// attach the texture to FBO color attachment point
		glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT,
								  GL_TEXTURE_2D, textureId, 0);

		// attach the renderbuffer to depth attachment point
		glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT,
									 GL_RENDERBUFFER_EXT, rboId);

		AllocateFBO = 1;
	}
	

	// check FBO status
	GLenum status = glCheckFramebufferStatusEXT( GL_FRAMEBUFFER_EXT );
		

	if( status != GL_FRAMEBUFFER_COMPLETE_EXT ) 
	{
		VerboseFBO(status);
		RenderFailed = 1;
	}
	else
	{
		float save_m_height = m_height;
		float save_m_width = m_width;
		m_height =  vsize_export;                     
		m_width = hsize_export; 

	 
		float MyRed   = (float)bg_red_export   / (float)255;
		float MyGreen = (float)bg_green_export / (float)255;
		float MyBlue  = (float)bg_blue_export  / (float)255;

		glClearColor( MyRed, MyGreen, MyBlue, 0.5f );

		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

		RenderSceneToOffScreen();

		RenderFBOToFile();

		m_height = save_m_height;                     
		m_width = save_m_width;

		glClearColor(m_color[BACKGROUND][0], m_color[BACKGROUND][1], m_color[BACKGROUND][2], 0.5f);
	}
}

}

void CGLView::RenderSceneToOffScreen()
{
	m_bBlackElement = 31337;

	// Renderscene pour opengl
	glLoadIdentity();
	glLightfv(GL_LIGHT0, GL_POSITION, light_position);

	SetPers(true);
	
	//Position de la scene
	glLoadIdentity();
	glPushMatrix();

	PanZoomRotate();
	glEnable(GL_DEPTH_TEST);

	BuildFont(m_CharSize);

	
	//Affichage des elements
	if( m_nType == 1 || m_nType == 2 )
	{
		RenderPrimitiv( 0, NLIST );
	}


	if(m_nType == 3)
	{
		// Primitives classiques
		RenderPrimitiv( 0, DIAG_N_MIN );
		
		// Primitives DIAG & ISO MIN
		RenderPrimitivMin( DIAG_N_MIN, SOUDURE );
		RenderPrimitivMin( ISO_DA_DX_MIN, DEFPL_INF );

		// Primitives DIAG & ISO MAX
		RenderPrimitivMax( DIAG_N_MAX, RELACHEMENT_SELECTED );
		RenderPrimitivMax( ISO_DA_DX_MAX, NLIST);
	}

	// Surcharge des elements en noir
	if(m_Affiche[NOEUDS]){Dessine(NOEUDS);}
	if(m_Affiche[NUMEROS_NOEUDS]){Dessine(NUMEROS_NOEUDS);}
	if(m_Affiche[ELEMENTS]){Dessine(ELEMENTS);}

	/*if(m_bSelectMode )
	{
		
		if(m_Affiche[ELEMENTS] || m_Affiche[SECTIONS_WIRE] || m_Affiche[SECTIONS_FULL])
		{		
				Dessine(ELEMENTS);
		}
		if(m_Affiche[CABLES])Dessine(CABLES);
		if(m_Affiche[DALLES_PLEINES] || m_Affiche[DALLES_CONTOUR])Dessine(DALLES_PLEINES);
		if(m_nType ==2)
		{
			if(m_Affiche[ELEMENTS_DEF] || m_Affiche[SECTIONS_WIRE_DEF] || m_Affiche[SECTIONS_FULL_DEF])
				Dessine(ELEMENTS_DEF);
			if(m_Affiche[CABLES_DEF])
				Dessine(CABLES_DEF);
			if(m_Affiche[DALLES_PLEINES_DEF] || m_Affiche[DALLES_CONTOUR_DEF])
				Dessine(DALLES_PLEINES_DEF);
		}
	}*/


	if(m_bValNum)
	{
		if(m_nType==2)
		{
			for(int i=DIAG_N;i<DIAG_CABLES;i++)
				if(m_Affiche[i])
					Dessine(i);
		}
		if(m_nType==3)
		{
			for(int i=DIAG_N_MIN;i<DIAG_N_MAX;i++)
				if(m_Affiche[i] && m_AfficheMin)
					Dessine(i);
			for(int i=DIAG_N_MAX;i<NLIST;i++)
				if(m_Affiche[i] && m_AfficheMax)
					Dessine(i);
		}
	}

	//pop pour PanZoomRotate
	glPopMatrix();

	if(m_bSelectMode && m_nSelectMode ==3 && m_state == PAN)
	{
		float x1,y1,x2,y2;
		MouseToGL(m_CurrentPoint,x1,y1);
		MouseToGL(m_Anchor,x2,y2);
		glColor3f(1.0f,1.0f,1.0f);
		glBegin(GL_LINE_LOOP);
		glVertex2f(x1,y1);glVertex2f(x2,y1);
		glVertex2f(x2,y2);glVertex2f(x1,y2);
		glEnd();
	}

	glDisable(GL_DEPTH_TEST); 
	SetPers(false);

	//dessin des axes
	glPushMatrix();
	glTranslatef(m_xmin+(m_xmax-m_xmin)*0.10,m_ymin+(m_ymax-m_ymin)*0.10,1.0f);
	glRotatef(m_rotx, 1.0f, 0.0f, 0.0f);
    glRotatef(m_roty, 0.0f, 1.0f, 0.0f);
   
	float facteurZoom = (m_xmax-m_xmin) * 0.02;
	glScalef(facteurZoom,facteurZoom,facteurZoom);
	DrawAxisOffScreen();
	glPopMatrix();
	
	// Ici que lon affiche l echelle d isocouleur
	// surement ici quil faut tester m_bimpose
	// afin de voir si on doit ou non recalculer les bornes d isocouleurs
	
	if(m_AfficheIso)
	{	

		if(m_pIsocoul!=NULL)
		{
			IsoEchelleOffScreen(m_pIsocoul);
		}

	}

	if(m_AfficheDiag)
	{
		LegendeDiag();
	}

	if(m_Affiche[CHARGES])
	{
		LegendeCharges();
	}

	if(m_nTit >0)
	{
		AfficheTitreOffScreen();
	}
	
  glFlush();
  m_bBlackElement = 1337;

}

void CGLView::RenderFBOToFile()
{

	glReadBuffer(GL_COLOR_ATTACHMENT0_EXT);


	glReadPixels( 0, 0, hsize_export ,  vsize_export , GL_BGR, GL_UNSIGNED_BYTE, ReadBufferGL );

	FILE *file = NULL;	    
	fopen_s( &file, path_export, "wb" );

	// Create & configure Bitmap and File info headers
	BITMAPFILEHEADER bitmapFileHeader;
	BITMAPINFOHEADER bitmapInfoHeader;

	bitmapFileHeader.bfType			= 0x4D42; //"BM" pour BMP
	bitmapFileHeader.bfSize			= vsize_export * hsize_export * 3;
	bitmapFileHeader.bfReserved1	= 0;
	bitmapFileHeader.bfReserved2	= 0;
	bitmapFileHeader.bfOffBits		= sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);
 
	bitmapInfoHeader.biSize			= sizeof(BITMAPINFOHEADER);
	bitmapInfoHeader.biWidth		= hsize_export ;
	bitmapInfoHeader.biHeight		= vsize_export ;
	bitmapInfoHeader.biPlanes		= 1;
	bitmapInfoHeader.biBitCount		= 24;
	bitmapInfoHeader.biCompression	= BI_RGB;
	bitmapInfoHeader.biSizeImage	= 0;
	bitmapInfoHeader.biXPelsPerMeter= 0; // ?
	bitmapInfoHeader.biYPelsPerMeter= 0; // ?
	bitmapInfoHeader.biClrUsed		= 0;
	bitmapInfoHeader.biClrImportant = 0;

    fwrite(&bitmapFileHeader, sizeof(BITMAPFILEHEADER), 1, file);
	fwrite(&bitmapInfoHeader, sizeof(BITMAPINFOHEADER), 1, file);
	fwrite(ReadBufferGL, vsize_export*hsize_export*3, 1, file);
	fclose(file);

}

void CGLView::SetPers(bool isModel)
{
	//Pour éviter une division par 0, les hauteurs et largeur
	//sont minimum 1
	if(m_height <= 0)  m_height = 1.f;                     
    if(m_width <= 0)   m_width = 1.f;                     
    
	glViewport(0, 0, m_width, m_height);

	//on réinitialise la matrice de projection
	glMatrixMode(GL_PROJECTION);            
	glLoadIdentity();                       

	float ratio = m_width/m_height;
	float zDim = m_zoom * 2.f;

	
	if(isModel && !m_bVueOrtho)
	{
		//perspective déformée -> angle de vue de 40 degré
		//gluPerspective(40, ratio, 0.01f, m_maxdim*8.f);
		gluPerspective(40, ratio, 1.0f, m_maxdim*8.f);
	}
	else
	{
		//on a le rapport dim = 1 (affichage écran sur le maximum pour zoom 1)
		//et m_maxdim/2.f (affichage écran pour m_zoom 1)
		if(m_longueur >= m_hauteur)
		{
			float dim = 1.f; //m_longueur/2.f;
			m_xmax = dim; 
			m_ymax = dim/ratio;  
		}
		else
		{
			float dim = 1.f; //m_hauteur/2.f;
			m_xmax = dim*ratio; 
			m_ymax = dim;   
		}

		m_xmin = -m_xmax;
		m_ymin = -m_ymax;

		if(isModel)
		{
			glOrtho(m_xmin, m_xmax, m_ymin, m_ymax, -zDim*4, (m_maxdim+zDim)*4 );
		}
		else
		{
			glOrtho(m_xmin, m_xmax, m_ymin, m_ymax, -40.f, 40.f);
		}
	}
	
	glMatrixMode(GL_MODELVIEW);             // Select the modelview matrix
	glLoadIdentity();                       // Reset the modelview matrix
}

Best regards ,

Tyrahell