Video and sound (sync'ed) in openGL

Hey people

Im currently working on a small game in openGL, with a couple of friends. The game part is comming out quite nicely, but we need cut-scenes of movie in the game. Unfortunately we have only been able to portay video material through applying textures (frame by frame). This is working somewhat all right, as long as only the video is portrayed. However we need audio as well. It’s no fun watching a movie cut-scene with badly sync’ed or no sound at all.

So please help us out here. We really need some inputs on how to attack this problem. And to make matters worse we are in a hurry. The game is for a school hand-in so time is not on our side.

OpenGL does not do sound, try SDL, fmod, bass, OpenAL, …
For sync to be correct, it is just a matter of timing properly the display of frames, pseudocode :


double elapsedTime = 0;
double startTime = now();
SDLStartSound(mysound);
while (video not ended) {
  drawFrameNumber(elapsedTime * VIDEO_FPS); // time in seconds,  otherwise fix with a multiplier
  swapbuffers;
  elapsedTime = now() - startTime;
}

If your video decoding only work sequentially, you will have to adapt drawFrameNumber(int x) like this:


global int currentFrame;
global img frame;
void drawFrameNumber(int x) {
  if (x< currentFrame) {
    // can't go back, draw current frame
    drawGlFrame(frame);
    return;
  }
  While (x > currentFrame) {
    decodeNextFrame(); // skip frame
    currentFrame++;
  }
  frame = decodeNextFrame();
  currentFrame++;
  drawGlFrame(frame);
}

Hey mate

Thanks for your input. Atm we are looking at the possibility of getting video and sound playback through DirectShow to work. It should in theory be possible, but we do not have it working in glut windows yet. Hopefully it will be possible.

Otherwise we will look into your suggestion!

I think that the problem has been solved now. I got the playback through DirectShow to work, and I put the playback into a glut window using the windows handle.

glutEnterGameMode();			//in window:	glutCreateWindow("title"); 
hwnd = FindWindow(L"GLUT", NULL);	//in window:	FindWindow(L"GLUT", "title");
printf("hwnd: %i
", hwnd);

This should show how I got the handle for my glut window (or as in my case my fullscreen enterGameMode “window”.

In my playBack class I did as follows:

//TO GET IT INTO ANOTHER WINDOW:
	m_videoWindow   = NULL;		//Get the video window

	hr = m_graphBuilder->QueryInterface(IID_IVideoWindow, (void **)&m_videoWindow);		//will return a IVideoWindow set of interfaces
	printf("hr - videoWindow: %i 
", hr);

	// Setup the video window to use our window handle
	hr = m_videoWindow->put_Owner((OAHWND)hwndINP);

	// Set the style of the video window
	hr = m_videoWindow->put_WindowStyle(WS_CHILD | WS_CLIPCHILDREN | WS_CLIPSIBLINGS);

	// Get the size of the window
	RECT rect;
	GetWindowRect(hwndINP, &rect);

// Set the video size:
	hr = m_videoWindow->SetWindowPosition(rect.left, rect.top, rect.right, rect.bottom);

hr is the HRESULT from DirectShow. All printf’s is just to check that everything runs as intended.

If anybody else needs cutscenes in their openGL game I hope that the above will help. Otherwise just ask if something is unclear!

EDIT: I forgot to say that it is not something that I made completely by myself. I copyed most from this link:

http://www.toymaker.info/Games/html/main_window_video.html

Just wanted to tell it so nobody thinks that Im trying to get credit for something another guy made! :slight_smile:

Glad you did it.
And thanks to have posted both your actual solution and source !