Part of the Khronos Group
OpenGL.org

The Industry's Foundation for High Performance Graphics

from games to virtual reality, mobile phones to supercomputers

Results 1 to 8 of 8

Thread: How to combine a point fragment shader with a blur fragment shader ?

  1. #1
    Junior Member Newbie
    Join Date
    Jun 2018
    Posts
    4

    How to combine a point fragment shader with a blur fragment shader ?

    Hi all,

    I'm trying like to display thousands of points on a 3D canvas (I'm using Processing) with a Depth of Field effect.
    More specifically, I would like to use a z-buffer (depth buffering) to adjust the level of blur of a `point` based on its distance from the camera.

    So far, I could come up with the following point shader:

    pointfrag.glsl

    Code :
    #ifdef GL_ES
        precision mediump float;
        precision mediump int;
        #endif
     
        varying vec4 vertColor;
        uniform float maxDepth;
     
        void main() {
     
          float depth = gl_FragCoord.z / gl_FragCoord.w;
          gl_FragColor = vec4(vec3(vertColor - depth/maxDepth), 1) ;
     
        }

    pointvert.glsl

    Code :
      uniform mat4 projection;
        uniform mat4 modelview;
     
        attribute vec4 position;
        attribute vec4 color;
        attribute vec2 offset;
     
     
        varying vec4 vertColor;
        varying vec4 vertTexCoord;
     
        void main() {
          vec4 pos = modelview * position;
          vec4 clip = projection * pos;
     
          gl_Position = clip + projection * vec4(offset, 0, 0);
     
          vertColor = color;
        }

    I also have a blur shader (originally from the PostFX library):

    blurfrag.glsl

    Code :
        #ifdef GL_ES
        precision mediump float;
        precision mediump int;
        #endif
     
     
        #define PROCESSING_TEXTURE_SHADER
     
        uniform sampler2D texture;
     
        // The inverse of the texture dimensions along X and Y
        uniform vec2 texOffset;
     
        varying vec4 vertColor;
        varying vec4 vertTexCoord;
     
        uniform int blurSize;       
        uniform int horizontalPass; // 0 or 1 to indicate vertical or horizontal pass
        uniform float sigma;        // The sigma value for the gaussian function: higher value means more blur
                                    // A good value for 9x9 is around 3 to 5
                                    // A good value for 7x7 is around 2.5 to 4
                                    // A good value for 5x5 is around 2 to 3.5
                                    // ... play around with this based on what you need <span class="Emoticon Emoticon1"><span>:)</span></span>
     
        const float pi = 3.14159265;
     
        void main() {  
          float numBlurPixelsPerSide = float(blurSize / 2); 
     
          vec2 blurMultiplyVec = 0 < horizontalPass ? vec2(1.0, 0.0) : vec2(0.0, 1.0);
     
          // Incremental Gaussian Coefficent Calculation (See GPU Gems 3 pp. 877 - 889)
          vec3 incrementalGaussian;
          incrementalGaussian.x = 1.0 / (sqrt(2.0 * pi) * sigma);
          incrementalGaussian.y = exp(-0.5 / (sigma * sigma));
          incrementalGaussian.z = incrementalGaussian.y * incrementalGaussian.y;
     
          vec4 avgValue = vec4(0.0, 0.0, 0.0, 0.0);
          float coefficientSum = 0.0;
     
          // Take the central sample first...
          avgValue += texture2D(texture, vertTexCoord.st) * incrementalGaussian.x;
          coefficientSum += incrementalGaussian.x;
          incrementalGaussian.xy *= incrementalGaussian.yz;
     
          // Go through the remaining 8 vertical samples (4 on each side of the center)
          for (float i = 1.0; i <= numBlurPixelsPerSide; i++) { 
            avgValue += texture2D(texture, vertTexCoord.st - i * texOffset * 
                                  blurMultiplyVec) * incrementalGaussian.x;         
            avgValue += texture2D(texture, vertTexCoord.st + i * texOffset * 
                                  blurMultiplyVec) * incrementalGaussian.x;         
            coefficientSum += 2.0 * incrementalGaussian.x;
            incrementalGaussian.xy *= incrementalGaussian.yz;
          }
     
          gl_FragColor = (avgValue / coefficientSum);
        }

    Question:

    - How can I combine the blur fragment shader with the point fragment shader ?

    Ideally I'd like to have one single fragment shader that computes the level of blur based on the z-coordinate of a point. Is that even possible ?

    Any help would be greatly appreciated.

    _________________________________________
    An example sketch displaying points using the pointfrag.glsl and pointvert.glsl shaders above:

    sketch.pde (PeasyCam library needed)

    Code :
    import peasy.*;
    import peasy.org.apache.commons.math.*;
    import peasy.org.apache.commons.math.geometry.*;
     
    PeasyCam cam;
    PShader pointShader;
    PShape shp;
    ArrayList<PVector> vectors = new ArrayList<PVector>();
     
    void setup() {
      size(900, 900, P3D);
      frameRate(1000);
      smooth(8);
     
      cam = new PeasyCam(this, 500);
      cam.setMaximumDistance(width);
      perspective(60 * DEG_TO_RAD, width/float(height), 2, 6000);
     
      double d = cam.getDistance()*3;
     
      pointShader = loadShader("pointfrag.glsl", "pointvert.glsl");
      pointShader.set("maxDepth", (float) d);
     
     
      for (int i = 0; i < 5000; i++) {
        vectors.add(new PVector(random(width), random(width), random(width)));
      }
     
      shader(pointShader, POINTS);
      strokeWeight(2);
      stroke(255);
     
      shp = createShape();
      shp.beginShape(POINTS);
      shp.translate(-width/2, -width/2, -width/2);  
      for (PVector v: vectors) {
        shp.vertex(v.x, v.y, v.z);
      }
      shp.endShape();
     
    }
     
    void draw(){
      background(0);
      shape(shp, 0, 0);
     
      cam.rotateY(.0001);
      cam.rotateX(.00005);
     
      println(frameRate);  
    }

    (see the sketch here: https://imgur.com/mf5dVRn)

    Click image for larger version. 

Name:	point_dof.jpg 
Views:	42 
Size:	12.5 KB 
ID:	2779


    [1]: https://github.com/cansik/processing.../blurFrag.glsl

  2. #2
    Senior Member OpenGL Guru
    Join Date
    Jun 2013
    Posts
    2,926
    Quote Originally Posted by solubi View Post
    Ideally I'd like to have one single fragment shader that computes the level of blur based on the z-coordinate of a point. Is that even possible ?
    Yes. Blurring points is somewhat simpler than applying a post-process depth-of-field effect to a scene.

    Enable GL_PROGRAM_POINT_SIZE. In a legacy or compatibility profile context, enable GL_POINT_SPRITE (in 3+ core profile, this is automatic). Enable GL_BLEND and use glBlendFunc(GL_SRC_ALPHA,GL_ONE). Disable GL_DEPTH_TEST.

    The vertex shader needs to set gl_PointSize according to the depth, so that the whole of the blur is within the bounds of the rendered point.

    The fragment shader just needs to calculate the opacity based upon gl_PointCoord. E.g.
    Code :
    float alpha = 1.0-smoothstep(0.0, 0.5, length(gl_PointCoord-0.5));

  3. #3
    Junior Member Newbie
    Join Date
    Jun 2018
    Posts
    4
    @GClements Thank you for your reply and suggestions. Unfortunately I'm not quite sure you're answering my question or maybe it's just me not understanding your answer.

    The fragment shader just needs to calculate the opacity based upon gl_PointCoord. E.g.
    The vertex shader needs to set gl_PointSize according to the depth
    In the sketch example I posted (GIF) the points already get darker as they get farther from the camera. They get smaller as well.

    I was asking for help to make a "mix" between the point fragment shader and the blur fragment shader. Here is my attempt:

    pointvert.glsl (just added 'vertTexCoord = vertTexCoord;' in 'main()')

    Code :
    uniform mat4 projection;
    uniform mat4 modelview;
     
    attribute vec4 position;
    attribute vec4 color;
    attribute vec2 offset;
     
     
    varying vec4 vertColor;
    varying vec4 vertTexCoord;
     
    void main() {
      vec4 pos = modelview * position;
      vec4 clip = projection * pos;
     
      gl_Position = clip + projection * vec4(offset, 0, 0);
     
      vertTexCoord = vertTexCoord;
     
      vertColor = color;
    }


    pointfrag.glsl (a mix between the original pointfrag.glsl and blurfrag.glsl)
    Code :
    #ifdef GL_ES
    precision mediump float;
    precision mediump int;
    #endif
     
    varying vec4 vertColor;
    uniform float maxDepth;
     
    uniform sampler2D texture;
    uniform vec2 texOffset; 
    varying vec4 vertTexCoord;
     
    uniform int blurSize;       
    uniform int horizontalPass; 
    uniform float sigma;                           
    const float pi = 3.14159265;
     
    void main() {
     
    float numBlurPixelsPerSide = float(blurSize / 2); 
     
      vec2 blurMultiplyVec = 0 < horizontalPass ? vec2(1.0, 0.0) : vec2(0.0, 1.0);
     
      vec3 incrementalGaussian;
      incrementalGaussian.x = 1.0 / (sqrt(2.0 * pi) * sigma);
      incrementalGaussian.y = exp(-0.5 / (sigma * sigma));
      incrementalGaussian.z = incrementalGaussian.y * incrementalGaussian.y;
     
      vec4 avgValue = vec4(0.0, 0.0, 0.0, 0.0);
      float coefficientSum = 0.0;
     
      avgValue += texture2D(texture, vertTexCoord.st) * incrementalGaussian.x;
      coefficientSum += incrementalGaussian.x;
      incrementalGaussian.xy *= incrementalGaussian.yz;
     
      for (float i = 1.0; i <= numBlurPixelsPerSide; i++) { 
        avgValue += texture2D(texture, vertTexCoord.st - i * texOffset * 
                              blurMultiplyVec) * incrementalGaussian.x;         
        avgValue += texture2D(texture, vertTexCoord.st + i * texOffset * 
                              blurMultiplyVec) * incrementalGaussian.x;         
        coefficientSum += 2.0 * incrementalGaussian.x;
        incrementalGaussian.xy *= incrementalGaussian.yz;
      }
     
     
      float depth = gl_FragCoord.z / gl_FragCoord.w;
      gl_FragColor = vec4(vec3(vertColor - depth/maxDepth), 1); // gl_FragColor from the point shader
      gl_FragColor =  (avgValue  / coefficientSum); // gl_FragColor from the blur shader
     
     
    }


    As you can see I don't know how to mix the depth information with the blur computation at the end of main()

    Code :
      float depth = gl_FragCoord.z / gl_FragCoord.w;
      gl_FragColor = vec4(vec3(vertColor - depth/maxDepth), 1); // gl_FragColor from the point shader
      gl_FragColor =  (avgValue  / coefficientSum); // gl_FragColor from the blur shader

    What I'd love to know:

    - Is this combination of the 2 fragment shader correct ?
    - How to mix the depth information with the blur computation in main() ?

    I hope I could make myself clear. Apologies if that still doesn't make sense for you.

  4. #4
    Intern Contributor
    Join Date
    Nov 2015
    Posts
    50
    If the only thing you'll be blurring is a point, then, rather than actually doing the blurs on-the-fly (proper Gaussian blurs are quite expensive!), I'd pre-compute all needed blur levels, save them as little textures, and simply display appropriate texture instead of the point.

  5. #5
    Junior Member Newbie
    Join Date
    Jun 2018
    Posts
    4
    @Utumno Thanks for your suggestion. For now I really want to try to mix the 2 fragment shaders and see for myself. I'll consider alternatives later.

  6. #6
    Junior Member Newbie
    Join Date
    Jun 2018
    Posts
    4
    I would really need your help to debug the pointfragment shader I wrote above.
    For example, I don't understand why the points aren't displayed when I combine the blufragment shader with the pointframent shader ? Shouln't it display blurred points at least ?

    pointvert.glsl

    Code :
    uniform mat4 projection;
    uniform mat4 modelview;
    uniform mat4 texMatrix;
     
     
    attribute vec4 position;
    attribute vec4 color;
    attribute vec2 offset;
    attribute vec2 texCoord;
     
     
     
    varying vec4 vertColor;
    varying vec4 vertTexCoord;
     
    void main() {
      vec4 pos = modelview * position;
      vec4 clip = projection * pos;
     
      gl_Position = clip + projection * vec4(offset, 0, 0);
     
      vertTexCoord = texMatrix * vec4(texCoord, 1.0, 1.0);
     
      vertColor = color;
    }

    pointfrag.glsl

    Code :
    #ifdef GL_ES
    precision mediump float;
    precision mediump int;
    #endif
     
    varying vec4 vertColor;
    uniform float maxDepth;
     
    uniform sampler2D texture;
    uniform vec2 texOffset; 
    varying vec4 vertTexCoord;
     
    uniform int blurSize;       
    uniform int horizontalPass; 
    uniform float sigma;                           
    const float pi = 3.14159265;
     
    void main() {
     
    float numBlurPixelsPerSide = float(blurSize / 2); 
     
      vec2 blurMultiplyVec = 0 < horizontalPass ? vec2(1.0, 0.0) : vec2(0.0, 1.0);
     
      vec3 incrementalGaussian;
      incrementalGaussian.x = 1.0 / (sqrt(2.0 * pi) * sigma);
      incrementalGaussian.y = exp(-0.5 / (sigma * sigma));
      incrementalGaussian.z = incrementalGaussian.y * incrementalGaussian.y;
     
      vec4 avgValue = vec4(0.0, 0.0, 0.0, 0.0);
      float coefficientSum = 0.0;
     
      avgValue += texture2D(texture, vertTexCoord.st) * incrementalGaussian.x;
      coefficientSum += incrementalGaussian.x;
      incrementalGaussian.xy *= incrementalGaussian.yz;
     
      for (float i = 1.0; i <= numBlurPixelsPerSide; i++) { 
        avgValue += texture2D(texture, vertTexCoord.st - i * texOffset * 
                              blurMultiplyVec) * incrementalGaussian.x;         
        avgValue += texture2D(texture, vertTexCoord.st + i * texOffset * 
                              blurMultiplyVec) * incrementalGaussian.x;         
        coefficientSum += 2.0 * incrementalGaussian.x;
        incrementalGaussian.xy *= incrementalGaussian.yz;
      }
     
     
      gl_FragColor =  (avgValue  / coefficientSum)  ;
     
     
    }
    Last edited by solubi; 06-06-2018 at 02:17 AM.

  7. #7
    Intern Contributor
    Join Date
    Nov 2015
    Posts
    50
    So when you run the above, all you see is an empty screen?

    I'd try to 'divide and conquer' the bug by removing code. First try commenting out everything from fragment shader's main() function and setting every pixel there to GREEN:

    Code :
    main()
    {
    gl_FragColor = vec4(0.0,1.0,0.0,1.0);
    }

    Can you see a GREEN screen now? If yes, the bug was inside frag shader's main(), if not - it's somewhere in vertex shader or the app...

  8. #8
    Intern Contributor
    Join Date
    Nov 2015
    Posts
    50
    The code is quite inefficient. For example this

    Code :
    vec3 incrementalGaussian;
      incrementalGaussian.x = 1.0 / (sqrt(2.0 * pi) * sigma);
      incrementalGaussian.y = exp(-0.5 / (sigma * sigma));
      incrementalGaussian.z = incrementalGaussian.y * incrementalGaussian.y;

    can be moved to the CPU, done only once and sent as a uniform (rather than re-doing it millions of times in each fragment shader).

    Actually what I recommend is pre-computing an array of blur offsets in CPU in a way described here:

    https://software.intel.com/en-us/blo...lur-algorithms

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •