Question: For example, we always assumed that the data or signal error is a Gaussian distribution? why?

I have asked this question on stackoverflow, the link: <https://stackoverflow.com/questions/12616406/anyone-can-tell-me-why-we-always-use-the-gaussian-distribution-in-machine-learni>

Answer: 
Machine learning (and statistics as well) treats data as the mix of deterministic (causal) and random parts. The random part of data usually has normal distribution. (Really, the causal relation is reverse: the distribution of random part of variable is called normal). Central limit theorem says that the sum of large number of varibles each having a small influence on the result approximate normal distribution. 
1. Why data is treated as normally distributed? In machine learning we want to express dependent variable as some function of a number of independent variables. If this function is sum (or expressed as a sum of some other funstions) and we are suggesting that the number of independent variables is really high, then the dependent variable should have normal distribution (due to central limit theorem).
2. Why errors are looked to be normally distributed? The dependent variable ($Y$) consists of deterministic and random parts. In machine learning we are trying to express deterministic part as a sum of deterministic independent variables: $$deterministic + random = func(deterministic(1))+...+func(deterministic(n))+model\\_error$$ If the whole deterministic part of $Y$ is explained by $X$ then the $model\\_error$ depicts only $random$ part, and thus should have normal distribution. So if error distribution is normal, then we may suggest that the model is successful. Else there are some other features that are absent in model, but have large enough influence on $Y$ (the model is incomplete) or the model is incorrect.
I had the same question "what the is advantage of doing a Gaussian transformation on predictors or target?" Infact, caret package has a pre-processing step that enables this transformation. 

I tried reasoning this out and am summarizing my understanding - 

1. Usually the data distribution in Nature follows a Normal distribution ( few examples like - age, income, height, weight etc., ) . So its the best approximation when we are not aware of the underlying distribution pattern.
2. Most often the goal in ML/ AI is to strive to make the data linearly separable even if it means projecting the data into higher dimensional space so as to find a fitting "hyperplane" (for example - SVM kernels, Neural net layers, Softmax etc.,). The reason for this being "Linear boundaries always help in reducing variance and is the most simplistic, natural and interpret-able" besides reducing mathematical / computational complexities. And, when we aim for linear separability, its always good to reduce the effect of outliers, influencing points and leverage points. Why? Because the hyperplane is very sensitive to the influencing points and leverage points (aka outliers) - To undertstand this - Lets shift to a 2D space where we have one predictor (X) and one target(y) and assume there exists a good positive correlation between X and y. Given this, if our X is normally distributed and y is also normally distributed, you are most likely to fit a straight line that has many points centered in the middle of the line rather than the end-points (aka outliers, leverage / influencing points). So the predicted regression line will most likely suffer little variance when predicting on unseen data.

Extrapolating the above understanding to a n-dimensional space and fitting a hyperplane to make things linearly separable does infact really makes sense because it helps in reducing the variance.