It’s a little more complicated than that.
Your program will never run on all computers. This is not an achievable goal. You need to decide where your reasonable cutoff point is.
OpenGL itself doesn’t have any modes that you can select. You get what the driver gives you and you have to accept it.
It’s getting increasingly difficult to find computers without some kind of graphics hardware installed. Even 5 year old business PCs will have graphics hardware that is capable of running simple OpenGL programs.
The worst case you’re going to see is Intel graphics chips. These are known for having bad OpenGL support, but their Direct3D support is OK (quite good in some cases, even). At that point you need to make a decision - if you must run on these you need to decide if you’re willing to switch to Direct3D and be able to run on Windows only, or stay with OpenGL and accept bad performance and limited functionality. That’s your choice to make and there is no best option for all cases; it depends on what your program needs to do and who your target audience is, and only you know that fully.
A general tip I give with Intel is that if a feature doesn’t exist in Direct3D then don’t expect it to work (well, or at all) in OpenGL. Getting to know some Direct3D is obviously useful here.
Working well can have two meanings. It might have bugs or other strange behaviours or it might run slowly (typically by being emulated in software).
In the case where OpenGL is software emulated, some or all of it may be done in software.
For simple programs, if the vertex pipeline is all that’s done in software then a fast enough CPU will be able to compensate and it will run fine.
If any of the per-fragment stages of the pipeline need to be done in software you’ll typically get less than 1 frame per second.
Not all OpenGL drivers are equal, and some are less equal than others. If your driver does not support some functionality then you cannot use it, not even in software. You’ll never be able to run GLSL programs on a TNT2 driver. You’ll never be able to use VBOs, you’ll never be able to use occlusion queries. You’re limited to what the TNT2 driver supports, and it doesn’t support these. This comes back to “do you really need to run on all computers?”
So two conditions need to be met. You hardware needs to support it. Your driver needs to support it. If these aren’t met, your driver might provide a software emulation. If your driver doesn’t provide a software emulation you can’t do it.
But… A full software implementation of OpenGL does exist (Mesa3D) (it’s not “official” and it’s not a “driver”, but it works) but this only implements OpenGL up to version 2.1 and is an all or nothing option. If you use it, everything will run in software, typically at less than 1 frame per second. You can’t mix and match parts of Mesa3D and parts of your driver, it’s one or the other.
It’s important to understand at this point that OpenGL is not software, and it’s not OpenGL that does the rendering. All that OpenGL does is convert OpenGL function calls to rendering commands (and data) in a format that your hardware can understand, then send them on to the hardware; your hardware is what does the actual rendering. Even in a full software emulation, the video hardware is still ultimately responsible for the final part which is plotting pixels on screen.
Worrying about driver quality at this stage is premature. No hardware vendor perfectly implements the standard, and you just have to live with it. NVIDIA will have the best support, Intel the worst and ATI/AMD somewhere inbetween. Driver quality will cause you some trouble, but avoid the common mistakes and use this forum when you have specific problems and you won’t go too far wrong.