Answers.Body: <p>I don't know how hard that would be to program with an AI but an AI would not just create a 3-d model from your single picture. Rather it would look through the vast collection of pictures in its training data looking for similar pictures (hopefully of similar objects) and then create a 3-d image from this large collection of pictures of (hopefully) the same object from different angles.</p>
  <p>If your single picture is of a reasonably common object this could still work very well but the moment you give it a picture of something not well represented in the training data it will fail miserably.</p>
  <p>So a picture of a watch will probably give you a good 3-d model but I would expect the model to fail on a tower made out of wooden toy bricks because every such tower looks different in 3-d.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-31T00:57:52.743Z
Answers.Text: Thanks for the reply. In this case, probably using AI isn't really the solution. The reason I assumed AI would be the solution for something like this is because I see things like Nvidia's DLSS or Photoshop's generative fill that use AI.
Answers.UserDisplayName: OGOG
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-30T06:46:46.023Z
Answers.LastActivityDate: 2023-08-30T06:46:46.023Z
Answers.UserDisplayName: quarague
Answers.Body: <p>I'm not sure if you're looking for an explanation on how one could train such model or for the tools to use it.</p>
  <p>If looking for tools :<br />
  paper: <a href="https://huggingface.co/papers/2309.16653" rel="nofollow noreferrer">https://huggingface.co/papers/2309.16653</a><br />
  tool: <a href="https://huggingface.co/spaces/jiawei011/dreamgaussian" rel="nofollow noreferrer">https://huggingface.co/spaces/jiawei011/dreamgaussian</a></p>
  <p>also: <a href="https://research.nvidia.com/labs/dir/magic3d/" rel="nofollow noreferrer">https://research.nvidia.com/labs/dir/magic3d/</a></p>
  <p>There are also some other companies offering this service but I have not tested them.</p>
  <p>If you would like to build your own tool I would suggest:</p>
  <ul>
  <li>Building a encoder-decoder like model</li>
  <li>Having a set of simple object with 3D model and multiple images to do photogrammetry</li>
  </ul>
  <p>Also state of the art seems more advanced in text-to-3D then 2d-to-3d so I would suggest to explore that first</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-10T12:29:55.050Z
Answers.Text: The `image-to-3d` models: https://huggingface.co/models?pipeline_tag=image-to-3d
Answers.UserDisplayName: Megidd
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-07T10:23:35.817Z
Answers.LastActivityDate: 2024-01-07T10:23:35.817Z
Answers.UserDisplayName: Timothee W
Body: <p>I'm posting this on the AI stack exchange because even though this can be solved with a &quot;regular&quot; complex and sophisticated algorithm, it seems that trying to generate something for which all information isn't available for is usually solved with AI techniques.</p>
  <p>The idea is to just have an image of some random object (e.g. a watch) and be able to generate a polished 3D model of it even though we don't have the full 360 view of it. This means there would have to be some inferences made, because the program would have infer what the rest of the object may look like.</p>
  <p>I would also like to know if this is an extremely complex endeavor, or maybe much easier than I think due to the recent advances in AI? There's a company, 3DFY.ai, that can turn a text prompt into a 3D model, so I would think that it's something along the lines of that, but using an image instead of text for reference.</p>
  <p>When doing a google search for the question that I'm asking, I get results that I'm not looking for, which are:</p>
  <ol>
  <li>Photogrammetry - this requires taking a picture from different angles of an object, which isn't always possible if you don't have access to the object.</li>
  <li>Depth Estimation - from what I've seen, it just looks like the image gets extruded a bit to give somewhat of a 3D appearance</li>
  </ol>
  <p>What kind of disciplines would this require besides AI? Computer Vision?</p>
  
Tags: <neural-networks><reinforcement-learning><machine-learning><deep-learning><computer-vision>
Title: How to generate a 3D model from only 1 image?
UserDisplayName: OGOG
ThreadText: # How to generate a 3D model from only 1 image?
  
  (Asked by: username_0 on 2023-08-30)
  
  I'm posting this on the AI stack exchange because even though this can be solved with a "regular" complex and sophisticated algorithm, it seems that trying to generate something for which all information isn't available for is usually solved with AI techniques.
  
  The idea is to just have an image of some random object (e.g. a watch) and be able to generate a polished 3D model of it even though we don't have the full 360 view of it. This means there would have to be some inferences made, because the program would have infer what the rest of the object may look like.
  
  I would also like to know if this is an extremely complex endeavor, or maybe much easier than I think due to the recent advances in AI? There's a company, 3DFY.ai, that can turn a text prompt into a 3D model, so I would think that it's something along the lines of that, but using an image instead of text for reference.
  
  When doing a google search for the question that I'm asking, I get results that I'm not looking for, which are:
  
  1. Photogrammetry - this requires taking a picture from different angles of an object, which isn't always possible if you don't have access to the object.
  2. Depth Estimation - from what I've seen, it just looks like the image gets extruded a bit to give somewhat of a 3D appearance
  
  What kind of disciplines would this require besides AI? Computer Vision?
  
  **Tags:** neural-networks, reinforcement-learning, machine-learning, deep-learning, computer-vision
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-08-30. Score: 1
  
  I don't know how hard that would be to program with an AI but an AI would not just create a 3-d model from your single picture. Rather it would look through the vast collection of pictures in its training data looking for similar pictures (hopefully of similar objects) and then create a 3-d image from this large collection of pictures of (hopefully) the same object from different angles.
  
  If your single picture is of a reasonably common object this could still work very well but the moment you give it a picture of something not well represented in the training data it will fail miserably.
  
  So a picture of a watch will probably give you a good 3-d model but I would expect the model to fail on a tower made out of wooden toy bricks because every such tower looks different in 3-d.
  
  > **Comments:**
  > **username_0**: Thanks for the reply. In this case, probably using AI isn't really the solution. The reason I assumed AI would be the solution for something like this is because I see things like Nvidia's DLSS or Photoshop's generative fill that use AI.  
  
  ---
  
  ### Answer by username_2 on 2024-01-07. Score: 1
  
  I'm not sure if you're looking for an explanation on how one could train such model or for the tools to use it.
  
  If looking for tools :  
  paper: <https://huggingface.co/papers/2309.16653>  
  tool: <https://huggingface.co/spaces/jiawei011/dreamgaussian>
  
  also: <https://research.nvidia.com/labs/dir/magic3d/>
  
  There are also some other companies offering this service but I have not tested them.
  
  If you would like to build your own tool I would suggest:
  
  * Building a encoder-decoder like model
  * Having a set of simple object with 3D model and multiple images to do photogrammetry
  
  Also state of the art seems more advanced in text-to-3D then 2d-to-3d so I would suggest to explore that first
  
  > **Comments:**
  > **username_3**: The `image-to-3d` models: https://huggingface.co/models?pipeline_tag=image-to-3d  
  
  ---
  
  