Answers.Body: <p>I don't know if my intuition is correct but I will give it a try. </p>
  
  <p>You could see weights as how much important one thing is, the problem is to understand what that thing represents. When I say <em>thing</em> I'm referring to the output of a specific neuron. I don't think that we can say what the output of a neuron represents in the real world unless we directly relate it through an error function or if the function used to compute that particular value have some meaning in the real world.</p>
  
  <p>Edit:</p>
  
  <p>If you want, you could actually build your neural network such that its neurons represent something. It's also very simple. you have only to write down all the equations relative to that particular topic. You could put them in a big system or, and this is better, you could put them in several systems such that the outputs of system 1 are the input of system 2 and so on. You could convert each system into a layer where each neuron represents an equation. Note that in this case, you would have the classical neuron with</p>
  
  <p><code>
  z = dot(w.T,x) + b
  a = g(x)
  </code></p>
  
  <p>but a more complex equation for <code>z</code> (but still based on weights) and a linear activation function for <code>a</code>. In this case, you could name each neuron and say what they represent in the real world. </p>
  
  <p>However, this isn't the purpose of a neural network. A neural network should have neurons with simple equations to be fast thus the linear interpolating function <code>dot(w.T,x) + b</code> is the best choice (the fact that the activation function is almost always non linear and in some cases a non-banal function is due to other thing and could be an interesting question). A neural network should also be as general as possible because usually is build upon a system that you don't know completely.</p>
  
  <p>So I modify slightly my answer: is not simply that you don't know what a neuron represent, excluding the ones of the output layer, you don't want that they have a meaning in the real world.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-19T03:57:48.397Z
Answers.Text: Thanks for your comment! It kinda validates my vague understanding into something more concrete. I would wonder if you know of any papers or citations that go into this topic?
Answers.UserDisplayName: user248884
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-20T10:49:31.250Z
Answers.Text: @user248884 Sorry but I don't have any paper on the topic. Also, this is my personal understanding of the topic and could be completely wrong. :)
Answers.UserDisplayName: gvgramazio
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-18T21:17:20.867Z
Answers.LastActivityDate: 2018-06-20T10:48:37.020Z
Answers.LastEditDate: 2018-06-20T10:48:37.020Z
Answers.UserDisplayName: gvgramazio
Answers.Body: <p>It's a bit of a challenge to answer your question, since you appear to be not really familiar with the basics. You're talking about mutations, and changes to the input. </p>
  
  <p>No. The input is a vector of data, which initializes the value of the input nodes. The first layer of weights is then used to calculate the values for the next layer of nodes. This next layer is not a "mutation" of the input layer; that suggest the second layer of nodes is similar but not exactly identical to the first layer. </p>
  
  <p>In reality, it's very common that the second layer of nodes does not even have the same <em>shape</em> as the first layer. </p>
  
  <p>You are even wondering if certain weights have a certain meaning. That's even easier to answer. We know these networks are quite robust. We can ignore a significant percentage of the weights, and the classifications will change only a little. This shows that no individual weight represents a specific aspect of the network. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-20T11:12:14.350Z
Answers.Text: Well, even if it's not as one should see it, is correct to say that each layer does a transformation on its input since it's a function that maps (n_i,m) -> (n_{i+1},m). Where m is the number of examples and n_i and n_{i+1} are the input feature and the output features.
Answers.UserDisplayName: gvgramazio
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-20T09:47:14.427Z
Answers.LastActivityDate: 2018-06-20T09:47:14.427Z
Answers.UserDisplayName: MSalters
Body: <p>I understand how neural networks work and have studied their theory well.</p>
  <p>My question is: On the whole, is there a clear understanding of how mutation occurs within a neural network from the input layer to the output layer, for both supervised and unsupervised cases?</p>
  <p>Any neural network is a set of neurons and connections with weights. With each successive layer, there is a change in the input. Say I have a neural network with <span class="math-container">$n$</span> parameters, which does movie recommendations. If <span class="math-container">$X$</span> is a parameter that stands for the movie rating on IMDB. In each successive stage, there is a mutation of input <span class="math-container">$X$</span> to <span class="math-container">$X'$</span> and further <span class="math-container">$X''$</span>, and so on.</p>
  <p>While we know how to mathematically talk about <span class="math-container">$X'$</span> and <span class="math-container">$X''$</span>, do we at all have a <strong>conceptual</strong> understanding as to what this variable is in its corresponding <span class="math-container">$n$</span>-dimensional parameter space?</p>
  <p>To the human eye, the neural network's weights might be a set of random numbers, but they may mean something profound, if we could ever understand what they 'represent'.</p>
  <p><a href="https://i.sstatic.net/JFcKa.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/JFcKa.jpg" alt="neural net" /></a></p>
  <p>What is the nature of the weights, such that, despite decades worth of research and use, there is no clear understanding of what these connection weights represent? Or rather, why has there been so little effort in understanding the nature of neural weights, in a non-mathematical sense, given the huge impetus in going beyond the black box notion of AI.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-17T10:26:40.497Z
Comments.Text: Hi, Thanks for sharing the post. I did make a reference of that post before I made this post. That post does go into a few tools that might help in seeing 'what' the weights are. The question here is a bit different though. I want to know what the fused neural network connection weight, say 0.7 would mean in some higher neural dimension? Is there any study on this?
Comments.UserDisplayName: user248884
Tags: <neural-networks><deep-learning><deep-neural-networks><weights><explainable-ai>
Title: What do the neural network's weights represent conceptually?
UserDisplayName: user248884
ThreadText: # What do the neural network's weights represent conceptually?
  
  (Asked by: username_0 on 2018-06-16)
  
  I understand how neural networks work and have studied their theory well.
  
  My question is: On the whole, is there a clear understanding of how mutation occurs within a neural network from the input layer to the output layer, for both supervised and unsupervised cases?
  
  Any neural network is a set of neurons and connections with weights. With each successive layer, there is a change in the input. Say I have a neural network with $n$ parameters, which does movie recommendations. If $X$ is a parameter that stands for the movie rating on IMDB. In each successive stage, there is a mutation of input $X$ to $X'$ and further $X''$, and so on.
  
  While we know how to mathematically talk about $X'$ and $X''$, do we at all have a **conceptual** understanding as to what this variable is in its corresponding $n$-dimensional parameter space?
  
  To the human eye, the neural network's weights might be a set of random numbers, but they may mean something profound, if we could ever understand what they 'represent'.
  
  [![neural net](https://i.sstatic.net/JFcKa.jpg)](https://i.sstatic.net/JFcKa.jpg)
  
  What is the nature of the weights, such that, despite decades worth of research and use, there is no clear understanding of what these connection weights represent? Or rather, why has there been so little effort in understanding the nature of neural weights, in a non-mathematical sense, given the huge impetus in going beyond the black box notion of AI.
  
  **Tags:** neural-networks, deep-learning, deep-neural-networks, weights, explainable-ai
  > **Comments:**
  > **username_0**: Hi, Thanks for sharing the post. I did make a reference of that post before I made this post. That post does go into a few tools that might help in seeing 'what' the weights are. The question here is a bit different though. I want to know what the fused neural network connection weight, say 0.7 would mean in some higher neural dimension? Is there any study on this?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-06-18. Score: 2
  
  I don't know if my intuition is correct but I will give it a try.
  
  You could see weights as how much important one thing is, the problem is to understand what that thing represents. When I say *thing* I'm referring to the output of a specific neuron. I don't think that we can say what the output of a neuron represents in the real world unless we directly relate it through an error function or if the function used to compute that particular value have some meaning in the real world.
  
  Edit:
  
  If you want, you could actually build your neural network such that its neurons represent something. It's also very simple. you have only to write down all the equations relative to that particular topic. You could put them in a big system or, and this is better, you could put them in several systems such that the outputs of system 1 are the input of system 2 and so on. You could convert each system into a layer where each neuron represents an equation. Note that in this case, you would have the classical neuron with
  
  `z = dot(w.T,x) + b
  a = g(x)`
  
  but a more complex equation for `z` (but still based on weights) and a linear activation function for `a`. In this case, you could name each neuron and say what they represent in the real world.
  
  However, this isn't the purpose of a neural network. A neural network should have neurons with simple equations to be fast thus the linear interpolating function `dot(w.T,x) + b` is the best choice (the fact that the activation function is almost always non linear and in some cases a non-banal function is due to other thing and could be an interesting question). A neural network should also be as general as possible because usually is build upon a system that you don't know completely.
  
  So I modify slightly my answer: is not simply that you don't know what a neuron represent, excluding the ones of the output layer, you don't want that they have a meaning in the real world.
  
  > **Comments:**
  > **username_0**: Thanks for your comment! It kinda validates my vague understanding into something more concrete. I would wonder if you know of any papers or citations that go into this topic?  
  > **username_1**: @user248884 Sorry but I don't have any paper on the topic. Also, this is my personal understanding of the topic and could be completely wrong. :)  
  
  ---
  
  ### Answer by username_2 on 2018-06-20. Score: 0
  
  It's a bit of a challenge to answer your question, since you appear to be not really familiar with the basics. You're talking about mutations, and changes to the input.
  
  No. The input is a vector of data, which initializes the value of the input nodes. The first layer of weights is then used to calculate the values for the next layer of nodes. This next layer is not a "mutation" of the input layer; that suggest the second layer of nodes is similar but not exactly identical to the first layer.
  
  In reality, it's very common that the second layer of nodes does not even have the same *shape* as the first layer.
  
  You are even wondering if certain weights have a certain meaning. That's even easier to answer. We know these networks are quite robust. We can ignore a significant percentage of the weights, and the classifications will change only a little. This shows that no individual weight represents a specific aspect of the network.
  
  > **Comments:**
  > **username_1**: Well, even if it's not as one should see it, is correct to say that each layer does a transformation on its input since it's a function that maps (n_i,m) -> (n_{i+1},m). Where m is the number of examples and n_i and n_{i+1} are the input feature and the output features.  
  
  ---
  
  