title: OpenGL: Read pixels from FrameBuffer for picking rounded up to 255 (0xFF)
body: <p>I am trying to implement object picking by packing the vao id to RGBA and render with it to an off-screen buffer which I then try to read with a buffer object.</p>
  
  <p>I am rendering to an off-screen buffer by creating a texture and a z-buffer RenderBuffer Object then attaching them to a FrameBuffer object as such:</p>
  
  <pre><code>/* create a framebuffer object */ 
  glGenFramebuffers(1, &amp;fbo);     
  /* attach the texture and the render buffer to the frame buffer */ 
  glBindFramebuffer(GL_FRAMEBUFFER, fbo); 
  
  /* generate a texture id */ 
  glGenTextures(1, &amp;tex); 
  /* bind the texture */ 
  glBindTexture(GL_TEXTURE_2D, tex); 
  /* create the texture in the GPU */ 
  glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, WINDOW_SIZE_X, WINDOW_SIZE_Y 
      , 0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr); 
  
  /* set texture parameters */ 
  glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); 
  glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); 
  glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); 
  glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); 
  
  /* unbind the texture */ 
  glBindTexture(GL_TEXTURE_2D, 0); 
  
  /* create a renderbuffer object for the depth buffer */ 
  glGenRenderbuffers(1, &amp;rbo); 
  /* bind the texture */ 
  glBindRenderbuffer(GL_RENDERBUFFER, rbo); 
  /* create the render buffer in the GPU */ 
  glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT 
      , WINDOW_SIZE_X, WINDOW_SIZE_Y); 
  
  /* unbind the render buffer */ 
  glBindRenderbuffer(GL_RENDERBUFFER, 0);
  
  
  /* attach the texture and the render buffer to the frame buffer */ 
  glFramebufferTexture(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, tex, 0); 
  glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT 
      , GL_RENDERBUFFER, rbo); 
  
  // check the frame buffer 
  if (glCheckFramebufferStatus( 
      GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) {
          std::cout &lt;&lt; "Framebuffer status not complete" &lt;&lt; '\n';
  }
  /* handle an error : frame buffer incomplete */ 
  /* return to the default frame buffer */ 
  glBindFramebuffer(GL_FRAMEBUFFER, 0); 
  </code></pre>
  
  <p>I also generate a pixel buffer object to read from the FrameBuffer after rendering is complete:</p>
  
  <pre><code>    /* generate the pixel buffer object */ 
  glGenBuffers(1,&amp;pbo);     
  glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo); 
  glBufferData(GL_PIXEL_PACK_BUFFER, WINDOW_SIZE_X * WINDOW_SIZE_Y * 4, nullptr, GL_STREAM_READ); 
  /* to avoid weird behaviour the first frame the data is loaded */ 
  glReadPixels(0, 0, WINDOW_SIZE_X, WINDOW_SIZE_Y, GL_BGRA, GL_UNSIGNED_BYTE, 0);     
  </code></pre>
  
  <p>Then in my render loop i bind it and render to that off-screen FrameBuffer:</p>
  
  <pre><code>    GLubyte red, green, blue, alpha; 
  
  /* bind the frame buffer */ 
  glBindFramebuffer(GL_FRAMEBUFFER, fbo); 
  
  /* clear the frame buffer */ 
  glClearColor(0,0,0,0); 
  glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); 
  
  /* select the shader program */ 
  glUseProgram(pickProgram); 
  /* set the object color */ 
  
  /*alpha = house.vaoId &amp; 0xFF; 
  blue = (house.vaoId &gt;&gt; 8) &amp; 0xFF; 
  green = (house.vaoId &gt;&gt; 16) &amp; 0xFF; 
  red = (house.vaoId &gt;&gt; 24) &amp; 0xFF; */
  
  GLuint objectId = 5;
  alpha = objectId &amp; 0xFF; 
  blue  = (objectId &gt;&gt; 8) &amp; 0xFF; 
  green = (objectId &gt;&gt; 16) &amp; 0xFF; 
  red   = (objectId &gt;&gt; 24) &amp; 0xFF; 
  
  //Upload the packed RGBA values to the shader   
  glUniform4f(baseColorUniformLocation, red, green ,blue, alpha);     
  
      //prepare to draw the object
  pvm = projectionMatrix*viewMatrix*house.modelMatrix;
  glUniformMatrix4fv(offScreenMatrixUniformLocation, 1, GL_FALSE, glm::value_ptr(pvm));
  
  /* draw the object*/ 
  glBindVertexArray(house.getVaoId());
  glDrawRangeElements(GL_TRIANGLES,0,42,42,GL_UNSIGNED_SHORT,NULL);
  glBindVertexArray(0);
  
  //check that our framebuffer is ok
  if(glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) {
  std::cout &lt;&lt; "Framebuffer Error" &lt;&lt; '\n';
  }
  
  GLuint temp;
      //get the object id from the read pixels
  temp = get_object_id(); &lt;--- this function is explained later
  
  
  /* return to the default frame buffer */
  glBindFramebuffer(GL_FRAMEBUFFER, 0); 
  </code></pre>
  
  <p>The fragment shader that renders to the offscreen buffer is very simple:</p>
  
  <pre><code>#version 420
  uniform vec4 BaseColor; 
  layout(location=0) out vec4 fragColor;
  
  void main() 
  { 
   fragColor = BaseColor; 
  } 
  </code></pre>
  
  <p>This is the function called during the off-screen rendering to extract the packaged RGBA from the framebuffer:</p>
  
  <pre><code>GLuint Engine::get_object_id() 
  { 
  static int frame_event = 0; 
  GLuint object_id; 
  int x, y; 
  GLuint red, green, blue, alpha, pixel_index; 
  //GLuint read_pbo, map_pbo;
  GLubyte* ptr; 
  
  /* read one pixel buffer */ 
  glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo_a); 
  /* map the other pixel buffer */  
      ///////////////////// NOTE :5th argument, BGRA or RGBA, doesn't make a difference right?
  glReadPixels(0, 0, WINDOW_SIZE_X, WINDOW_SIZE_Y, GL_BGRA, GL_UNSIGNED_BYTE, 0);
  ptr = (GLubyte*)glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_WRITE); 
  /* get the mouse coordinates */ 
  /* OpenGL has the {0,0} at the down-left corner of the screen */ 
  glfwGetMousePos(&amp;x, &amp;y); 
  y = WINDOW_SIZE_Y - y; 
  object_id = -1; 
  if (x &gt;= 0 &amp;&amp; x &lt; WINDOW_SIZE_X &amp;&amp; y &gt;= 0 &amp;&amp; y &lt; WINDOW_SIZE_Y){ 
  //////////////////////////////////////////
      //I have to admit I don't understand what he does here
  ///////////////////////////////////////////
      pixel_index = (x + y * WINDOW_SIZE_X) * 4; 
      blue = ptr[pixel_index]; 
      green = ptr[pixel_index + 1]; 
      red = ptr[pixel_index + 2]; 
      alpha = ptr[pixel_index + 3]; 
  
  object_id = alpha +(red &lt;&lt; 24) + (green &lt;&lt; 16) + (blue &lt;&lt; 8);
  } 
  glUnmapBuffer(GL_PIXEL_PACK_BUFFER); 
  glBindBuffer(GL_PIXEL_PACK_BUFFER, 0); 
  return object_id; 
  } 
  </code></pre>
  
  <p>The problem is that this final bit of code that is supposed to read the pixels from the off-screen Framebuffer and give me the vao_id, has the following weird behavior:</p>
  
  <p>If ANY of the 4 bytes packed into the RGBA (sent via the shader) is anything other than 0 that byte comes out as 0xFF at the other end.</p>
  
  <p>So if I send</p>
  
  <pre><code>00000000 00000000 00000000 00000001 
  </code></pre>
  
  <p>I will get</p>
  
  <pre><code>00000000 00000000 00000000 11111111 
  </code></pre>
  
  <p>Or if I send</p>
  
  <pre><code>00000000 00010000 00000000 00000000 
  </code></pre>
  
  <p>I will get</p>
  
  <pre><code>00000000 11111111 00000000 00000000 
  </code></pre>
  
  <p>...when I read the pixels with get_object_id().</p>
  
  <p>If I bind the texture and render it to a quad on my normal FrameBuffer the colors that I pass to the off-screen rendering come out correct on the quad. But the pixels read by the get_object_id() make every byte sent in rounded up to 255 (0xFF). So my guess is there's a problem with that final function.</p>