No ! I think you misunderstand some things. OpenGL is a library, implemented for several languages (ranged from C up to Visual Basic). A language is (almost) just a mean in order to interface between the programmer and the machine.
Nowadays, GL allows us to directly use the GPU for performing some graphic stuffs (and even if it’s not graphical). Noone (I guess) will never be able to do some stuffs on the GPU without using GL (or Dx). When you program with any language, you program the CPU not the GPU. Actually, the single exception is from using shaders (GLSL, HLSL, cG) but they all are inherited from GL (that’s a library).
Still from the older days, anyone was able to do some 3D: this was not a limit from the language or the library but was limited by the computer performances only. Now the computers are more performant this changes many things, but really that’s not a language that prevents someone from doing any 3D. (of course, some languages are inherited from their system, so some are slowler, some are fast…).
If you don’t want OpenGL for doing 3D, don’t use it ! But since Geforce series, your applications will be slowler than those using T&L, shaders and so (so threw GL or Dx).
If you want some limits, they all came from different kind of things: what you want to do, how fast you need it to perform, how real you want it to look and so on. If you want to make the new Quake 2, then I’m pretty sure you can do it without using GL at all, just by doing some maths on the CPU. This is the same for industrial applications: if the simulation is really more important that the look, then GL might not suit your needs.
I reread your last question. And I guess you might be right. If we look well, shaders are replacing default GL commands (Translate,Rotate…) but shaders are still part of GL. I guess this is the aim: beeing able to program whether the CPU or the GPU. Using normal ways (C,C++,Java…) will do stuffs on CPU, and using shaders will do work on GPU.
Hope I answered your questions.