Body: <p>The light field of a certain scene is the set of all light rays that travel through the volume of that scene at a specific point in time. A light field camera, for example, captures and stores a subset of the light field of a scene.</p>
  <p>I've got an unstructured subsampling of such a scene (a few billion rays, each having a direction and light intensity information).</p>
  <p>What I wish to do now is to create an approximation of the original scene that created this light field, with the approximation consisting of 3 arbitrarily positioned (alpha-)textured 2D planes (in 3D space), where each point on the surface radiates light towards uniformly in all directions based on the pixel color at this position.</p>
  <p>So, I guess, this is like finding regions in the volume where similarly 'colored' rays intersect, such that the planes maximize the number of intersections they can cover.</p>
  <p>So, the available data is the few billions of rays, the desired output is the parameters(position, normal and size) of the three planes plus one RGBA texture for each.</p>
  <p>I'm asking here about experiences and opinions: Is this problem rather well-suited for a machine learning approach or rather not?</p>
  <p>Edit:</p>
  <p>A classical algorithm I could think of to solve this would be to voxelize the volume and use pathtracing to add a color sample for each ray to all cells along its way, then give each cell some value based on how similar all its contained samples are and then search for planar surfaces that intersect as many high rated cells as possible.</p>
  <p>But maybe machine learning is better suited for such a problem?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-07T12:00:01.020Z
Comments.Text: Do you have more data than just direction and intensity? To define the trajectory inside a volume you also need some reference location that the ray is projecting to or from - e.g. its intersection point with the scene/volume.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-07T13:36:28.917Z
Comments.Text: Oh, of course! I do have the point of impact of each ray (NOT the point of origin! think as if the whole scene was an image sensor. So, I know from what direction the lightrays come from and where they impact as well as the color/intensity of the source, but not how far away this source is along the ray
Comments.UserDisplayName: matthias_buehlmann
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-07T15:25:50.817Z
Comments.Text: also some of the lightrays originate outside of the volume for which I have impact points
Comments.UserDisplayName: matthias_buehlmann
Tags: <machine-learning><algorithm-request>
Title: Is this a problem well suited for machine learning?
UserDisplayName: matthias_buehlmann
ThreadText: # Is this a problem well suited for machine learning?
  
  (Asked by: username_0 on 2020-01-07)
  
  The light field of a certain scene is the set of all light rays that travel through the volume of that scene at a specific point in time. A light field camera, for example, captures and stores a subset of the light field of a scene.
  
  I've got an unstructured subsampling of such a scene (a few billion rays, each having a direction and light intensity information).
  
  What I wish to do now is to create an approximation of the original scene that created this light field, with the approximation consisting of 3 arbitrarily positioned (alpha-)textured 2D planes (in 3D space), where each point on the surface radiates light towards uniformly in all directions based on the pixel color at this position.
  
  So, I guess, this is like finding regions in the volume where similarly 'colored' rays intersect, such that the planes maximize the number of intersections they can cover.
  
  So, the available data is the few billions of rays, the desired output is the parameters(position, normal and size) of the three planes plus one RGBA texture for each.
  
  I'm asking here about experiences and opinions: Is this problem rather well-suited for a machine learning approach or rather not?
  
  Edit:
  
  A classical algorithm I could think of to solve this would be to voxelize the volume and use pathtracing to add a color sample for each ray to all cells along its way, then give each cell some value based on how similar all its contained samples are and then search for planar surfaces that intersect as many high rated cells as possible.
  
  But maybe machine learning is better suited for such a problem?
  
  **Tags:** machine-learning, algorithm-request
  > **Comments:**
  > **username_1**: Do you have more data than just direction and intensity? To define the trajectory inside a volume you also need some reference location that the ray is projecting to or from - e.g. its intersection point with the scene/volume.  
  > **username_0**: Oh, of course! I do have the point of impact of each ray (NOT the point of origin! think as if the whole scene was an image sensor. So, I know from what direction the lightrays come from and where they impact as well as the color/intensity of the source, but not how far away this source is along the ray  
  > **username_0**: also some of the lightrays originate outside of the volume for which I have impact points  
  
  ---
  
  ## 0 answers
  
  