Part of the Khronos Group
OpenGL.org

The Industry's Foundation for High Performance Graphics

from games to virtual reality, mobile phones to supercomputers

Results 1 to 2 of 2

Thread: opengl ray picking/tracing with cardboard (GVR)

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Newbie Newbie
    Join Date
    Dec 2017
    Posts
    2

    opengl ray picking/tracing with cardboard (GVR)

    For the demonstration, I made a small application that draws three squares of random size and random location. Drawing squares:
    Code java:
    private void renderObject(float[] modelViewProjection, FloatBuffer vertexCoordsBuf, int i) {
            GLES20.glUseProgram(mProgram);
            GLES20.glEnableVertexAttribArray(mPositionHandle);
            GLES20.glVertexAttribPointer(
                    mPositionHandle, 3,
                    GLES20.GL_FLOAT, false,
                    12, vertexCoordsBuf);
     
     
            GLES20.glUniform4fv(mColorHandle,
                    1, colors, i * 4);
     
            GLES20.glUniformMatrix4fv(mMVPMatrixHandle,
                    1, false, modelViewProjection, 0);
     
            // Draw the square
            GLES20.glDrawElements(
                    GLES20.GL_TRIANGLES, 6,
                    GLES20.GL_UNSIGNED_SHORT, indexesBuffer);
            // Disable vertex array
            GLES20.glDisableVertexAttribArray(mPositionHandle);
    }

    I also display the cursor (white small square) in the center of the screen using the orthogonal projection:

    Code java:
    private void renderCursor(Eye eye) {
            GLES20.glDepthFunc(GLES20.GL_ALWAYS);
            GLES20.glDepthMask(false);
     
            GLES20.glUseProgram(mProgram);
            GLES20.glEnableVertexAttribArray(mPositionHandle);
            GLES20.glVertexAttribPointer(
                    mPositionHandle, 3,
                    GLES20.GL_FLOAT, false,
                    12, cursorVertexCoordsBuf);
     
     
            GLES20.glUniform4fv(mColorHandle,
                    1, colors, 12);
     
            float[] perspectiveMatrix = eye.getPerspective(Z_NEAR, Z_FAR);
            float[] orthoViewMatrix = new float[16];
            float[] modelViewMatrix = new float[16];
            Matrix.orthoM(orthoViewMatrix, 0, -1, 1, -1, 1, Z_NEAR, Z_FAR);
            Matrix.multiplyMM(modelViewMatrix, 0, perspectiveMatrix, 0, orthoViewMatrix, 0);
     
            GLES20.glUniformMatrix4fv(mMVPMatrixHandle,
                    1, false, modelViewMatrix, 0);
     
     
            // Draw the square
            GLES20.glDrawElements(
                    GLES20.GL_TRIANGLES, 6,
                    GLES20.GL_UNSIGNED_SHORT, indexesBuffer);
            // Disable vertex array
            GLES20.glDisableVertexAttribArray(mPositionHandle);
            GLES20.glDepthFunc(GLES20.GL_LEQUAL);
            GLES20.glDepthMask(true);
        }

    For clarity in the center of the visible area for stereo mode, I draw red dots. To see how the cursor deviates from the center of the screen.

    You can see screenshot in attachment. How you can see white cursor moved from centers of screens: How it see in stereo mode

    If I have disabled the stereo mode then everything works fine. I use a method to determine which object in the center of the screen I'm producing ray tracing for screenWidth/2 and screenHeight/2 coordinates:

    Code java:
    public boolean rayPicking(int viewWidth, int viewHeight, float rx, float ry,
                                  float[] viewMatrix, float[] projMatrix, float[] modelViewMatrix,
                                  float[] objectCoords, short[] objectIndexes) {
            float[] near_xyz = unProject(rx, ry, 0, viewMatrix, projMatrix, viewWidth, viewHeight);
            float[] far_xyz = unProject(rx, ry, 1, viewMatrix, projMatrix, viewWidth, viewHeight);
            int coordCount = objectCoords.length;
            float[] convertedSquare = new float[coordCount];
            float[] resultVector = new float[4];
            float[] inputVector = new float[4];
            for (int i = 0; i < coordCount; i = i + 3) {
                inputVector[0] = objectCoords[i];
                inputVector[1] = objectCoords[i + 1];
                inputVector[2] = objectCoords[i + 2];
                inputVector[3] = 1;
                Matrix.multiplyMV(resultVector, 0, modelViewMatrix, 0, inputVector, 0);
                convertedSquare[i] = resultVector[0] / resultVector[3];
                convertedSquare[i + 1] = resultVector[1] / resultVector[3];
                convertedSquare[i + 2] = resultVector[2] / resultVector[3];
            }
     
     
            ArrayList<Triangle> triangles = new ArrayList<>();
            for (int i = 0; i < objectIndexes.length; i = i + 3) {
                int i1 = objectIndexes[i] * 3;
                int i2 = objectIndexes[i + 1] * 3;
                int i3 = objectIndexes[i + 2] * 3;
                triangles.add(
                        new Triangle(
                                new float[]{
                                        convertedSquare[i1],
                                        convertedSquare[i1 + 1],
                                        convertedSquare[i1 + 2]
                                },
                                new float[]{
                                        convertedSquare[i2],
                                        convertedSquare[i2 + 1],
                                        convertedSquare[i2 + 2]
                                },
                                new float[]{
                                        convertedSquare[i3],
                                        convertedSquare[i3 + 1],
                                        convertedSquare[i3 + 2]
                                }
                        )
                );
            }
     
            for (Triangle t : triangles) {
                float[] point = new float[3];
                int intersects = Triangle.intersectRayAndTriangle(near_xyz, far_xyz, t, point);
                if (intersects == 1 || intersects == 2) {
                    return true;
                }
            }
            return false;
    }

    But if i turn on the stereo mode, the cursor is not displayed in the center of the screen. And for the left eye is moved in the left from the center of the visible area of ​​the left eye, and for the right is appropriately to the right. And in this case, I do not understand how to get the real screen coordinates for the cursor so that it can produce ray tracing? I will be glad to any prompts

    Full worked sample you can get from github: https://github.com/Anton111111/Andro...yPickingSample

    And all placed in one file to easy read it: https://github.com/Anton111111/Andro...nActivity.java
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	1.jpg 
Views:	9 
Size:	26.7 KB 
ID:	2568  
    Last edited by Dark Photon; 12-08-2017 at 06:07 AM.

  2. #2
    Newbie Newbie
    Join Date
    Dec 2017
    Posts
    2
    I fixed it. Now my ray picking works well in stereo mode. I add calculateCursorScreenCoords method to get screen coordinates of cursor. https://github.com/Anton111111/Andro...vity.java#L196

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •