Answers.Body: <blockquote>
    <p>Why use images of reads when you already have the reads data?</p>
  </blockquote>
  
  <p>Neural networks take one or multiple <a href="https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.ndarray.html" rel="nofollow noreferrer">N-dimension arrays</a> (n-d arrays) as input; they can't take arbitrary forms of data. To feed reads and their alignment into the network, you have to encode them as n-d arrays. If you ignore "colors" for now, a straightforward choice is a (windowLength, readDepth) 2D-array as you see in an alignment browser – it is an image. The "colors" of this image are often called "channels" or "features" in deep learning (DL). They represent base types and base quality in deepVariant. In the end, you get a (windowLength, readDepth, nFeatures) 3D array as input. This is the most natural encoding to DL researchers.</p>
  
  <blockquote>
    <p>Why chose CNN for a variant caller</p>
  </blockquote>
  
  <p>That is because CNN has been repeatedly proved to be the best architecture for such (height, width, nChannels) 3D arrays as input. Read alignments are exactly that.</p>
  
  <hr>
  
  <p>On a historical note, deepVariant was first built on top of google's now-deprecated <a href="http://www.techtimes.com/articles/104696/20151109/google-announces-machine-learning-system-tensorflow-now-open-sourced.htm" rel="nofollow noreferrer">DisBelief framework</a> that only takes RGB images as input. They had to squeeze all types of information into three color channels. Now with TensorFlow, an image can have an arbitrary number of color channels. This simplifies implementation and improves accuracy. In addition to this image, deepVariant also takes other information as input.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-19T22:47:29.980Z
Answers.LastActivityDate: 2018-04-19T22:47:29.980Z
Answers.UserDisplayName: user172818
Answers.Body: <p>It's more "images of reads" rather than actual images of reads. In reality, they're feeding in a pileup, which is just a matrix of letters or numbers (you can visualize this as an image, of course). This is more apparent if you look at <a href="https://github.com/google/deepvariant/blob/r0.6/docs/visualizing_examples.ipynb" rel="nofollow noreferrer">one of their example python notebooks</a>, CNNs are popular in the image processing field, but at the end of the day images are just large matrices of values so any time you can convert your input into that sort of format then a CNN can be useful (provided you have sufficient training data and some GPUs to throw at things).</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-30T17:05:15.850Z
Answers.Text: I find the answer confusing: are you saying that the CNN is *not* fed images? Because the way google describes DeepVariant, they are indeed creating pileup image data and putting that through their CNN.
Answers.UserDisplayName: Konrad Rudolph
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-30T18:05:53.007Z
Answers.Text: It's not an image, it's an "image". The DeepVariant are a bit ambiguous on that point, though at the end of the day it's a multidimensional matrix of values (the relation of that to an image being obvious).
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-19T14:37:20.753Z
Answers.LastActivityDate: 2018-04-19T14:37:20.753Z
Answers.UserDisplayName: Devon Ryan
Body: <p>Google released their variant caller <a href="https://github.com/google/deepvariant" rel="nofollow noreferrer">DeepVariant</a> which won the highest SNP performance award in the <a href="https://precision.fda.gov/challenges/truth/results" rel="nofollow noreferrer">Precision FDA Truth challenge</a> (99.999% accuracy).</p>
  
  <p>From the linked github repo, we see that DeepVariant is a CNN, we provide images of the reads as an input and train the neural network with a stochastic gradient descent algorithm. </p>
  
  <p>To me this is unintuitive. Why use images of reads when you already have the reads data?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-19T14:31:02.840Z
Comments.Text: https://en.wikipedia.org/wiki/Law_of_the_instrument
Comments.UserDisplayName: heathobrien
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-19T14:38:33.127Z
Comments.Text: Where did you see they need images? The [documentation](https://github.com/google/deepvariant/blob/r0.6/docs/deepvariant-details.md) says the input is a bam file, just like any other caller.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-19T14:50:14.860Z
Comments.Text: @terdon Images (of pileups, of course) are mentioned [in the preprint](https://www.biorxiv.org/content/early/2018/03/20/092890). I think they're also using the Inception CNN, which is heavily used in image processing. I think many people forget that images are just matrices.
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-19T14:50:31.887Z
Comments.Text: @terdon It seems that the first version was on images: `We have made a number of improvements to the methodology as well. The biggest change was to move away from RGB-encoded (3-channel) pileup images and instead represent the aligned read data using a multi-channel tensor data layout.` and now they use something else. Thank you!
Comments.UserDisplayName: Claudiu Creanga
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-21T20:14:13.997Z
Comments.Text: As @DevonRyan has already said, these are not images per se. From a machine's point of view an image is just a bunch of matrices (one per colour channel). I'd say this is just an example of bad/confusing use of terminology.
Comments.UserDisplayName: Eli Korvigo
Tags: <variant-calling>
Title: Why chose CNN for a variant caller
ThreadText: # Why chose CNN for a variant caller
  
  (Asked by: username_0 on 2018-04-19)
  
  Google released their variant caller [DeepVariant](https://github.com/google/deepvariant) which won the highest SNP performance award in the [Precision FDA Truth challenge](https://precision.fda.gov/challenges/truth/results) (99.999% accuracy).
  
  From the linked github repo, we see that DeepVariant is a CNN, we provide images of the reads as an input and train the neural network with a stochastic gradient descent algorithm.
  
  To me this is unintuitive. Why use images of reads when you already have the reads data?
  
  **Tags:** variant-calling
  
  > **Comments:**
  > **username_1**: https://en.wikipedia.org/wiki/Law_of_the_instrument  
  > **username_2**: Where did you see they need images? The [documentation](https://github.com/google/deepvariant/blob/r0.6/docs/deepvariant-details.md) says the input is a bam file, just like any other caller.  
  > **username_3**: @username_2 Images (of pileups, of course) are mentioned [in the preprint](https://www.biorxiv.org/content/early/2018/03/20/092890). I think they're also using the Inception CNN, which is heavily used in image processing. I think many people forget that images are just matrices.  
  > **username_0**: @username_2 It seems that the first version was on images: `We have made a number of improvements to the methodology as well. The biggest change was to move away from RGB-encoded (3-channel) pileup images and instead represent the aligned read data using a multi-channel tensor data layout.` and now they use something else. Thank you!  
  > **username_4**: As @username_3 has already said, these are not images per se. From a machine's point of view an image is just a bunch of matrices (one per colour channel). I'd say this is just an example of bad/confusing use of terminology.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2018-04-19. Score: 4 | Accepted answer
  
  It's more "images of reads" rather than actual images of reads. In reality, they're feeding in a pileup, which is just a matrix of letters or numbers (you can visualize this as an image, of course). This is more apparent if you look at [one of their example python notebooks](https://github.com/google/deepvariant/blob/r0.6/docs/visualizing_examples.ipynb), CNNs are popular in the image processing field, but at the end of the day images are just large matrices of values so any time you can convert your input into that sort of format then a CNN can be useful (provided you have sufficient training data and some GPUs to throw at things).
  
  > **Comments:**
  > **username_6**: I find the answer confusing: are you saying that the CNN is *not* fed images? Because the way google describes DeepVariant, they are indeed creating pileup image data and putting that through their CNN.  
  > **username_3**: It's not an image, it's an "image". The DeepVariant are a bit ambiguous on that point, though at the end of the day it's a multidimensional matrix of values (the relation of that to an image being obvious).  
  
  ---
  
  ### Answer by username_5 on 2018-04-19. Score: 3
  
  > Why use images of reads when you already have the reads data?
  
  Neural networks take one or multiple [N-dimension arrays](https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.ndarray.html) (n-d arrays) as input; they can't take arbitrary forms of data. To feed reads and their alignment into the network, you have to encode them as n-d arrays. If you ignore "colors" for now, a straightforward choice is a (windowLength, readDepth) 2D-array as you see in an alignment browser – it is an image. The "colors" of this image are often called "channels" or "features" in deep learning (DL). They represent base types and base quality in deepVariant. In the end, you get a (windowLength, readDepth, nFeatures) 3D array as input. This is the most natural encoding to DL researchers.
  
  > Why chose CNN for a variant caller
  
  That is because CNN has been repeatedly proved to be the best architecture for such (height, width, nChannels) 3D arrays as input. Read alignments are exactly that.
  
  ---
  
  On a historical note, deepVariant was first built on top of google's now-deprecated [DisBelief framework](http://www.techtimes.com/articles/104696/20151109/google-announces-machine-learning-system-tensorflow-now-open-sourced.htm) that only takes RGB images as input. They had to squeeze all types of information into three color channels. Now with TensorFlow, an image can have an arbitrary number of color channels. This simplifies implementation and improves accuracy. In addition to this image, deepVariant also takes other information as input.
  
  
  ---
  
  