I am trying to do multi-gpu work on Linux, using specifically 2 k4200 quadro’s. I have attempted to use SLI, but even after changing the X11/xorg.conf file using nvidia-xconfig for SLI to be SFR, it doesn’t seem to be using the 2nd GPU (glmark2 tests I’ve tried remain in the same ballpark). So first question is how do I enable SLI for these cards in Linux? Is it even possible?
If it’s not possible, I was wondering if I can manually split the work & perform offscreen rendering on each GPU. However, is there a Linux/X11 equivalent of wglEnumGpusNV to get list of available GPU’s? Also I was hoping to use glXCopyImageSubDataNV to copy data from one GPU to the other. Does that data transfer go through the CPU (i.e. slow) or does it go directly between the GPU’s?
For the answer to the first, I’m sure there are a lot of ways to skin that cat. But one way is to set up each GPU on a separate X screen (:0.1, :0.2) in your xorg.conf. Then in your app, talk to these two screens (GPUs) in two different threads (one screen per thread). That is, in each thread Open an X connection to the appropriate screen, then create a GL context to use for talking to that screen.
Thanks for the reply and the link. I was able to get a separate gl context for each gpu if I connect 2 displays (monitors), one to each gpu. However, is there anyway to have just a single display/monitor but have 2 x11 screens so I can create separate gl contexts for each? I am playing around with the xorg.conf file and am unable to get a 2nd x screen created on the 2nd GPU (unless I hook up a 2nd monitor). Any thoughts?
Well, what you have set up is multiple X screens on a single X display (:0.0 and :0.1, where the notation is :<display>.<screen>). But I suspect you really meant two separate X screens (each driven by their own GPU) on the same physical monitor.
As far as I know, this isn’t supported working like this out-of-the-box without custom software. The video output needs to be scanned out by one of the cards. And for one card to scan out a frame rendered on the other, it’d need to be transferred to it from the other GPU somehow. You could do this in software though.
Another approach is to feed the output of the two GPUs into a video combiner of some sort, which itself would then scan out the composite signal. However, depending on your use case, you may need to synchronize the scan-out of the two GPUs’ video signals. This is what G-Sync on Quadro offers (last time I looked into it).
If you don’t need logically distinct, separately-addressable GPUs and just want them to work together to render a frame on a single monitor, use NVidia SLI mode.
I would recommend doing some reading in:
/usr/share/doc/NVIDIA_GLX-1.0/README.txt
It tends to have pretty good documentation on what is and isn’t supported with the NVidia GL drivers. In particular, I’d focus on these sections:
Hi, thanks for the response again. I wanted to try SLI, but it doesn’t seem to be working for 2 Quadro k4200’s? Nvidia driver picks up both the video cards, but each time I try to set SLI mode to SFR in X11, it doesn’t work, and if I dump xorg.conf from nvidia-settings GUI, it sets SLI back to off.
[QUOTE=Dark Photon;1279726]Well, what you have set up is multiple X screens on a single X display (:0.0 and :0.1, where the notation is :<display>.<screen>). But I suspect you really meant two separate X screens (each driven by their own GPU) on the same physical monitor.
[/QUOTE]
Also to clarify, I don’t really care about the output for the 2nd GPU. I only need the first GPU to be connected to a monitor. The 2nd GPU doesn’t need (and shouldn’t) be connected to that same monitor. It’s just headless.